Data center anomaly analysis method and system

Abstract

The embodiment of the present application provides a data center abnormality analysis method and system, and the present application provides a data center operation abnormality analysis and prediction method. First, operation data of a target data center is acquired. Then, the operation data of the target data center is loaded into an abnormality analysis and prediction network generated through prior learning, and the abnormality analysis and prediction network is generated based on knowledge learning of associated data center operation data and static scheduling data. Through the abnormality analysis and prediction network, a corresponding operation abnormality category can be generated. Finally, abnormality diagnosis data is determined according to the determined operation abnormality category. This method can effectively analyze and predict possible abnormal conditions in the operation process of the data center, and provides strong support for the management and maintenance of the data center.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data centers, in particular to a data center anomaly analysis method and system. BACKGROUND

[0002] With the development of information technology, data centers, as the core facilities for storing and processing large amounts of data, their running state is crucial to the stability of the entire IT system. However, due to the complexity of data centers and the variability of the operating environment, various types of abnormal situations may occur during operation, such as hardware failure, software error, network interruption, etc.

[0003] Traditional anomaly detection methods usually rely on manual monitoring or pre-set threshold alarms, which have certain limitations. For example, manual monitoring is inefficient and prone to errors; the pre-set threshold method cannot adapt to the dynamic changes of the data center operating environment, and may produce a large number of false positives or false negatives.

[0004] In addition, when an anomaly occurs in a data center, it is necessary to quickly and accurately diagnose the fault to minimize the service interruption time caused by the fault. However, current fault diagnosis methods often need to rely on the experience of professional maintenance personnel, and when faced with complex or unknown types of anomalies, they may not be able to make effective diagnoses.

[0005] Therefore, there is an urgent need for a new method that can effectively analyze and predict abnormal situations during the operation of a data center, and quickly and accurately diagnose faults according to the abnormal category. SUMMARY

[0006] Therefore, the purpose of the embodiments of the present application is to provide a data center anomaly analysis method and system, and the present application provides a method for analyzing and predicting data center operation anomalies. First, the operating data of the target data center is obtained. Then, the operating data of the target data center is loaded into an anomaly analysis and prediction network generated through prior learning, and the anomaly analysis and prediction network is generated based on the knowledge learning of the associated data center operating data and static scheduling data. Through the anomaly analysis and prediction network, the corresponding operating anomaly category can be generated. Finally, according to the determined operating anomaly category, the anomaly diagnosis data is determined. This method can effectively analyze and predict abnormal situations that may occur during the operation of a data center, providing strong support for the management and maintenance of data centers.

[0007] According to an aspect of the embodiments of the present application, a data center anomaly analysis method and system are provided, the method comprising:

[0008] obtaining target data center operating data;

[0009] loading the target data center operation data into the prior learning abnormal analysis prediction network to generate an operation abnormality category determined by the abnormal analysis prediction network, wherein the abnormal analysis prediction network is generated based on knowledge learning of associated data center operation data and static scheduling data;

[0010] determining abnormal diagnosis data according to the operation abnormality category.

[0011] In an alternative embodiment, the abnormal analysis prediction network comprises a self-attention unit and a fully connected output unit, and the loading of the target data center operation data into the prior learning abnormal analysis prediction network to generate an operation abnormality category determined by the abnormal analysis prediction network comprises:

[0012] loading the target data center operation data into the self-attention unit to generate a target self-attention feature determined by the self-attention unit;

[0013] loading the target self-attention feature into the fully connected output unit to generate an operation abnormality category determined by the fully connected output unit.

[0014] In an alternative embodiment, the training step of the abnormal analysis prediction network comprises:

[0015] generating positive learning features and negative learning features based on the linked operation scheduling event data in the data center log data, and generating template learning data based on the positive learning features and the negative learning features;

[0016] updating the parameters of the initialized abnormal analysis prediction network based on the template learning data to generate an abnormal analysis prediction network with updated parameters.

[0017] In an alternative embodiment, the generating of the positive learning features and the negative learning features based on the linked operation scheduling event data in the data center log data comprises:

[0018] obtaining the linked operation scheduling event data in the data center log data as basic training template data;

[0019] performing rule-based conversion on the basic training template data to generate the positive learning features;

[0020] randomly shuffling the data center operation data and noise features in the data center log data to generate the negative learning features.

[0021] In an alternative implementation, the self-attention unit includes a first self-attention unit and a second self-attention unit, and the parameter updating of the initialized abnormality analysis prediction network according to the template learning data generates a parameter-updated abnormality analysis prediction network, including:

[0022] loading the static scheduling event of the operation scheduling event pair into the first self-attention unit to generate a target static self-attention feature determined by the first self-attention unit, and loading the dynamic scheduling event of the operation scheduling event pair into the second self-attention unit to generate a target dynamic self-attention feature determined by the second self-attention unit;

[0023] determining a target training error parameter according to the target static self-attention feature and the target dynamic self-attention feature, and performing parameter updating on the self-attention unit to generate a parameter-updated self-attention unit, with the target training error parameter minimized as the goal;

[0024] performing parameter updating on the fully connected output unit according to the parameter-updated self-attention unit to generate a parameter-updated fully connected output unit.

[0025] In an alternative implementation, the operation scheduling event pair includes a plurality of dynamic scheduling events, and the loading of the dynamic scheduling event of the operation scheduling event pair into the second self-attention unit to generate the target dynamic self-attention feature determined by the second self-attention unit includes:

[0026] integrating each of the dynamic scheduling events to generate integrated data center operation data;

[0027] loading the integrated data center operation data into the second self-attention unit to generate the target dynamic self-attention feature determined by the second self-attention unit.

[0028] In an alternative implementation, the operation scheduling event pair includes a plurality of dynamic scheduling events, and the loading of the dynamic scheduling event of the operation scheduling event pair into the second self-attention unit to generate the dynamic self-attention feature determined by the second self-attention unit includes:

[0029] loading each of the dynamic scheduling events into the second self-attention unit to generate a dynamic self-attention feature of each of the dynamic scheduling events determined by the second self-attention unit;

[0030] summing the dynamic self-attention features of each of the dynamic scheduling events to generate the target dynamic self-attention feature.

[0031] In an alternative implementation, the first self-attention unit and the second self-attention unit are respectively connected with the full connection output unit, and the self-attention unit after the parameter update updates the parameters of the full connection output unit to generate a full connection output unit after the parameter update, comprising:

[0032] For the static scheduling data in the template learning data, the first self-attention unit is used to determine the static self-attention feature of the static scheduling data;

[0033] The target abnormal learning template data is constructed according to the static self-attention feature and the abnormal category of the static scheduling data;

[0034] The initialized full connection output unit is updated according to the target abnormal learning template data to generate a full connection output unit after the parameter update.

[0035] According to another aspect of the embodiment of the present application, a data center abnormality analysis method and system are provided, and the system comprises:

[0036] An acquisition module is configured to acquire target data center operation data;

[0037] A generation module is configured to load the target data center operation data into an abnormality analysis prediction network after prior learning to generate an operation abnormality category determined by the abnormality analysis prediction network, which is generated by the abnormality analysis prediction network after knowledge learning according to associated data center operation data and static scheduling data;

[0038] A determination module is configured to determine abnormality diagnosis data according to the operation abnormality category.

[0039] According to another aspect of the embodiment of the present application, a server is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete the communication among each other through the communication bus; the memory is configured to store a computer program; and the processor is configured to execute the computer program to implement the steps of the data center abnormality analysis method.

[0040] According to another aspect of the embodiment of the present application, a readable storage medium is provided, and the readable storage medium stores a computer program, which can execute the steps of the data center abnormality analysis method when the computer program is run by a processor.

[0041] In order to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the following will be combined with embodiments and the accompanying drawings for detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0043] Figure 1 The component schematic diagram of the server provided by the embodiment of the present application is shown;

[0044] Figure 2 The flow schematic diagram of the data center anomaly analysis method provided by the embodiment of the present application is shown;

[0045] Figure 3 The functional module block diagram of the data center anomaly analysis system provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0046] In order to make the students in the technical field better understand the present application scheme, the following will combine the drawings in the embodiments of the present application, and the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. According to the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0047] The terms "first", "second", "third" and the like (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0048] Figure 1An exemplary component diagram of server 100 is shown. Server 100 can include one or more processors 104, such as one or more central processing units (CPUs), each of which can implement one or more hardware threads. Server 100 can also include any storage medium 106, which is used to store any kind of information, such as code, settings, data, etc. Without limitation, for example, storage medium 106 can include any one or combination of the following: any type of RAM, any type of ROM, a flash memory device, a hard disk, an optical disk, etc. More generally, any storage medium can use any technology for storing information. Further, any storage medium can provide volatile or non-volatile retention of information. Further, any storage medium can represent a fixed or removable component of server 100. In one case, server 100 can perform any of the operations described herein when processor 104 executes associated instructions stored in any storage medium or combination of storage media. Server 100 also includes one or more drive units 108, such as hard drive units, optical drive units, etc., for interfacing with any storage medium.

[0049] Server 100 also includes input / output 110 (I / O), which is used to receive various inputs (via input unit 112) and to provide various outputs (via output unit 114). One particular output mechanism can include a presentation device 116 and associated graphical user interface (GUI) 118. Server 100 can also include one or more network interfaces 120, which are used to exchange data with other devices via one or more communication units 122. One or more communication buses 124 couple the above-described components together.

[0050] Communication unit 122 can be implemented in any manner, such as through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. Communication unit 122 can include any combination of hardwired links, wireless links, routers, gateway functionality, name servers, etc., governed by any protocol or combination of protocols.

[0051] Figure 2 A flow diagram of a data center anomaly analysis method and system provided by embodiments of the present application is shown. The data center anomaly analysis method and system can be performed by server 100 shown in FIG. 1, and the detailed steps of the data center anomaly analysis method are introduced as follows. Figure 1

[0052] Step S110, obtaining target data center operation data;

[0053] ​Step S120, load the target data center operation data into the prior learning anomaly analysis prediction network to generate an operation anomaly category determined by the anomaly analysis prediction network, which is generated based on knowledge learning of the associated data center operation data and static scheduling data;

[0054] Step S130, determine abnormal diagnosis data according to the operation anomaly category.

[0055] Based on the above steps, the embodiment provides an analysis and prediction method for data center operation anomalies. First, the operation data of a target data center is obtained. Then, the operation data of the target data center is loaded into an anomaly analysis prediction network generated by prior learning, and the anomaly analysis prediction network is generated based on knowledge learning of the associated data center operation data and static scheduling data. The corresponding operation anomaly category can be generated through the anomaly analysis prediction network. Finally, abnormal diagnosis data is determined according to the determined operation anomaly category. This method can effectively analyze and predict possible anomalies in the operation process of a data center, and provides strong support for the management and maintenance of the data center.

[0056] In an alternative implementation, the anomaly analysis prediction network includes a self-attention unit and a fully connected output unit, and the loading of the target data center operation data into the prior learning anomaly analysis prediction network to generate the operation anomaly category determined by the anomaly analysis prediction network includes:

[0057] loading the target data center operation data into the self-attention unit to generate target self-attention features determined by the self-attention unit;

[0058] loading the target self-attention features into the fully connected output unit to generate an operation anomaly category determined by the fully connected output unit.

[0059] In an alternative implementation, the training step of the anomaly analysis prediction network includes:

[0060] generating positive learning features and negative learning features according to the linked operation scheduling event data in the data center log data, and generating template learning data according to the positive learning features and the negative learning features;

[0061] updating the parameters of the initialized anomaly analysis prediction network according to the template learning data to generate an anomaly analysis prediction network with updated parameters.

[0062] In an alternative implementation, the generation of positive learning features and negative learning features according to the linked operation scheduling event data in the data center log data includes:

[0063] Obtaining linkage running scheduling event data in data center log data as basic training template data;

[0064] Converting the basic training template data by rules to generate the positive learning feature;

[0065] Randomly shuffling data center running data and noise features in the data center log data to generate the negative learning feature.

[0066] In an alternative embodiment, the self-attention unit includes a first self-attention unit and a second self-attention unit, and the parameter updating of the initialized abnormality analysis prediction network according to the template learning data to generate the parameter-updated abnormality analysis prediction network includes:

[0067] For a running scheduling event data pair in the template learning data, loading a static scheduling event of the running scheduling event data pair into the first self-attention unit to generate a target static self-attention feature determined by the first self-attention unit, and loading a dynamic scheduling event of the running scheduling event data pair into the second self-attention unit to generate a target dynamic self-attention feature determined by the second self-attention unit;

[0068] Determining a target training error parameter according to the target static self-attention feature and the target dynamic self-attention feature, and updating the parameter of the self-attention unit to generate a parameter-updated self-attention unit, with the minimum of the target training error parameter as the goal;

[0069] Updating the parameter of the fully connected output unit according to the parameter-updated self-attention unit to generate a parameter-updated fully connected output unit.

[0070] In an alternative embodiment, the running scheduling event data pair includes a plurality of dynamic scheduling events, and the loading of the dynamic scheduling event of the running scheduling event data pair into the second self-attention unit to generate the target dynamic self-attention feature determined by the second self-attention unit includes:

[0071] Integrating each of the dynamic scheduling events to generate integrated data center running data;

[0072] Loading the integrated data center running data into the second self-attention unit to generate the target dynamic self-attention feature determined by the second self-attention unit.

[0073] In an alternative implementation, the operation scheduling event data pair contains a plurality of dynamic scheduling events, and loading the dynamic scheduling events of the operation scheduling event data pair into the second self-attention unit to generate dynamic self-attention features determined by the second self-attention unit includes:

[0074] loading each of the dynamic scheduling events into the second self-attention unit to generate dynamic self-attention features of each of the dynamic scheduling events determined by the second self-attention unit;

[0075] summing the dynamic self-attention features of each of the dynamic scheduling events to generate the target dynamic self-attention feature.

[0076] In an alternative implementation, the first self-attention unit and the second self-attention unit are respectively connected to the fully connected output unit, and the self-attention unit after parameter update is used to update the parameters of the fully connected output unit to generate a fully connected output unit after parameter update, including:

[0077] determining static self-attention features of static scheduling data in the template learning data according to the first self-attention unit;

[0078] constructing target abnormal learning template data according to the static self-attention features and the abnormal categories of the static scheduling data;

[0079] updating the parameters of the initialized fully connected output unit according to the target abnormal learning template data to generate a fully connected output unit after parameter update.

[0080] Figure 3 The function module diagram of the data center anomaly analysis system 200 according to the embodiment of the present application is shown, and the functions implemented by the data center anomaly analysis system 200 can correspond to the steps of the above method. The data center anomaly analysis system 200 can be understood as the above-mentioned server 100, or the processor of the server 100, and can also be understood as a component independent of the above-mentioned server 100 or the processor, which realizes the functions of the present application under the control of the server 100, such as Figure 3 As shown, the functions of each function module of the data center anomaly analysis system 200 will be described in detail below.

[0081] The acquisition module 210 is configured to acquire target data center operation data.

[0082] The generating module 220 is configured to load the target data center operation data into the prior learning anomaly analysis prediction network to generate an operation anomaly category determined by the anomaly analysis prediction network, wherein the anomaly analysis prediction network is generated according to knowledge learning of associated data center operation data and static scheduling data.

[0083] The determining module 230 is configured to determine anomaly diagnosis data according to the operation anomaly category.

[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein.

[0085] It is apparent for those skilled in the art that the present application is not limited to the details of the foregoing exemplary embodiments, but can be implemented in other concrete forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the foregoing description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application.

Claims

1. A data center anomaly analysis method, characterized by, The method comprises: acquiring target data center operation data; loading the target data center operation data into a priori learning anomaly analysis prediction network to generate an operation anomaly category determined by the anomaly analysis prediction network, which is generated by knowledge learning according to associated data center operation data and static scheduling data; determining abnormal diagnosis data according to the operation anomaly category; The anomaly analysis prediction network comprises a self-attention unit and a fully connected output unit, and loading the target data center operation data into the priori learning anomaly analysis prediction network to generate the operation anomaly category determined by the anomaly analysis prediction network comprises: loading the target data center operation data into the self-attention unit to generate target self-attention features determined by the self-attention unit; loading the target self-attention features into the fully connected output unit to generate an operation anomaly category determined by the fully connected output unit; The training step of the anomaly analysis prediction network comprises: generating positive learning features and negative learning features according to linked operation scheduling event data in data center log data, and generating template learning data according to the positive learning features and the negative learning features; updating parameters of the initialized anomaly analysis prediction network according to the template learning data to generate an anomaly analysis prediction network with updated parameters; The self-attention unit comprises a first self-attention unit and a second self-attention unit, and updating parameters of the initialized anomaly analysis prediction network according to the template learning data to generate an anomaly analysis prediction network with updated parameters comprises: loading static scheduling events of the operation scheduling event data pair into the first self-attention unit to generate target static self-attention features determined by the first self-attention unit, and loading dynamic scheduling events of the operation scheduling event data pair into the second self-attention unit to generate target dynamic self-attention features determined by the second self-attention unit for the operation scheduling event data pair in the template learning data; determining a target training error parameter according to the target static self-attention features and the target dynamic self-attention features, and updating parameters of the self-attention unit to generate a self-attention unit with updated parameters, with the objective of minimizing the target training error parameter; updating parameters of the fully connected output unit according to the self-attention unit with updated parameters to generate a fully connected output unit with updated parameters; The operation scheduling event data pair comprises a plurality of dynamic scheduling events, and loading the dynamic scheduling events of the operation scheduling event data pair into the second self-attention unit to generate target dynamic self-attention features determined by the second self-attention unit comprises: integrating each of the dynamic scheduling events to generate integrated data center operation data; loading the integrated data center operation data into the second self-attention unit to generate target dynamic self-attention features determined by the second self-attention unit; The operation scheduling event data pair contains a plurality of dynamic scheduling events, and the loading of the dynamic scheduling events of the operation scheduling event data pair into the second self-attention unit generates dynamic self-attention features determined by the second self-attention unit, including: Respectively loading each of the dynamic scheduling events into the second self-attention unit generates dynamic self-attention features of each of the dynamic scheduling events determined by the second self-attention unit; Summing the dynamic self-attention features of each of the dynamic scheduling events generates the target dynamic self-attention feature; The first self-attention unit and the second self-attention unit are respectively connected with the full connection output unit, and the parameter updated self-attention unit updates the parameters of the full connection output unit to generate a parameter updated full connection output unit, including: For the static scheduling data in the template learning data, the first self-attention unit determines the static self-attention features of the static scheduling data; According to the static self-attention features and the abnormal class of the static scheduling data, a target abnormal learning template data is constructed; According to the target abnormal learning template data, the initialized full connection output unit is updated to generate a parameter updated full connection output unit.

2. The data center anomaly analysis method of claim 1, wherein, The generation of positive learning features and negative learning features according to the linked operation scheduling event data in the data center log data includes: Obtaining the linked operation scheduling event data in the data center log data as basic training template data; The basic training template data is converted by a rule to generate the positive learning features; The data center operation data and noise features in the data center log data are randomly shuffled to generate the negative learning features.

3. A data center anomaly analysis system, characterized by, Including: An acquisition module is configured to acquire target data center operation data; A generation module is configured to load the target data center operation data into a priori learned abnormal analysis prediction network to generate an operation abnormal class determined by the abnormal analysis prediction network; A determination module is configured to determine abnormal diagnosis data according to the operation abnormal class; The abnormal analysis prediction network includes a self-attention unit and a full connection output unit, and the loading of the target data center operation data into the priori learned abnormal analysis prediction network to generate the operation abnormal class determined by the abnormal analysis prediction network includes: The target data center operation data is loaded into the self-attention unit to generate target self-attention features determined by the self-attention unit; The target self-attention features are loaded into the full connection output unit to generate an operation abnormal class determined by the full connection output unit; The training process of the abnormal analysis prediction network includes: According to the linked operation scheduling event data in the data center log data, positive learning features and negative learning features are generated, and template learning data is generated according to the positive learning features and the negative learning features; According to the template learning data, the initialized anomaly analysis prediction network is updated in parameters to generate an anomaly analysis prediction network after parameter update; The self-attention unit includes a first self-attention unit and a second self-attention unit, and the updating of the initialized anomaly analysis prediction network in parameters according to the template learning data to generate an anomaly analysis prediction network after parameter update includes: For a running scheduling event data pair in the template learning data, a static scheduling event of the running scheduling event data pair is loaded into the first self-attention unit to generate a target static self-attention feature determined by the first self-attention unit, and a dynamic scheduling event of the running scheduling event data pair is loaded into the second self-attention unit to generate a target dynamic self-attention feature determined by the second self-attention unit; According to the target static self-attention feature and the target dynamic self-attention feature, a target training error parameter is determined, and the self-attention unit is updated in parameters to generate a self-attention unit after parameter update; According to the self-attention unit after parameter update, the fully connected output unit is updated in parameters to generate a fully connected output unit after parameter update; The running scheduling event data pair includes a plurality of dynamic scheduling events, and the loading of the dynamic scheduling event of the running scheduling event data pair into the second self-attention unit to generate the target dynamic self-attention feature determined by the second self-attention unit includes: The dynamic scheduling events are integrated to generate integrated data center operation data; The integrated data center operation data is loaded into the second self-attention unit to generate the target dynamic self-attention feature determined by the second self-attention unit; The running scheduling event data pair includes a plurality of dynamic scheduling events, and the loading of the dynamic scheduling event of the running scheduling event data pair into the second self-attention unit to generate the dynamic self-attention feature determined by the second self-attention unit includes: The dynamic scheduling events are integrated to generate integrated data center operation data; The dynamic scheduling events are integrated to generate integrated data center operation data; The first self-attention unit and the second self-attention unit are respectively connected with the fully connected output unit, and the updating of the fully connected output unit in parameters according to the self-attention unit after parameter update to generate a fully connected output unit after parameter update includes: For the static scheduling data in the template learning data, a static self-attention feature of the static scheduling data is determined according to the first self-attention unit; According to the static self-attention feature and an abnormality category of the static scheduling data, a target abnormality learning template data is constructed; According to the target abnormality learning template data, the initialized fully connected output unit is updated in parameters to generate a fully connected output unit after parameter update.

4. A server, characterized by includes: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory accomplish mutual communication through the communication bus; The memory is used for storing a computer program; and the processor is used for executing the computer program to realize the steps of the data center anomaly analysis method in any one of claims 1-2.

Images