A virtualization system monitoring method, system, device and storage medium

Abstract

The present invention proposes a virtualization system monitoring method, system, device and storage medium, which includes configuring the first monitoring item and first monitoring data of the initial monitoring strategy, and calculating the normal distribution expectation and standard deviation of the first monitoring data; performing data association on the first monitoring data to generate association rules between the monitoring indicators of the first monitoring item; randomly generating a number of monitoring strategies, establishing a mapping between the monitoring strategies and the corresponding monitoring data as training samples of the neural network and inputting them into the neural network, and outputting decision information after training of the neural network to generate trained monitoring strategies. Based on this method, a virtualization system monitoring system, device and storage medium are also proposed. The present invention helps users discover the correlation between monitoring items based on big data applications, automatically formulates monitoring strategies according to the actual situation of the system, sets thresholds, and helps users automatically generate monitoring strategies according to system operation needs, thereby reducing resource consumption of non-correlated monitoring items.

Description

Technical Field

[0001] The present invention belongs to the technical field of virtualization system monitoring, and in particular relates to a virtualization system monitoring method, system, device and storage medium. Background Art

[0002] A virtualization system is an application service that creates one or more virtual computer systems on a physical computer system. With the growing development of cloud computing, user applications are becoming increasingly microservice-oriented and granular. Consequently, the monitoring systems for virtualization systems that provide the infrastructure resources for cloud computing are becoming increasingly complex.

[0003] Traditional monitoring systems collect and record monitoring indicators in real time, such as CPU usage, memory usage, storage and network I / O rates for hosts and virtual machines. The more monitoring items and indicators collected and the more frequently they are recorded, the more helpful they are for tracking operational issues. However, as the number of monitoring items increases and the frequency of metric collection increases, the non-functional consumption of virtualization system resources increases. In severe cases, this can even impact normal user operations, such as blocking storage I / O writes. To reduce resource consumption in the monitoring system, many virtualization systems have deleted monitoring items to facilitate future tracking of system operational issues. Traditional monitoring systems are often only able to respond to individual monitoring items and user-set thresholds. Threshold settings rely on the experience of operational maintenance personnel or other personnel and fail to objectively and comprehensively reflect the actual operating status of the current virtualization system. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes a virtualization system monitoring method, system, device and storage medium, which improves the virtualization monitoring capability by integrating big data, neural networks and traditional monitoring.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A virtualization system monitoring method includes the following steps:

[0007] Configuring a first monitoring item and first monitoring data of an initial monitoring strategy, and calculating a normal distribution expectation and a standard deviation of the first monitoring data; performing data association on the first monitoring data to generate association rules between monitoring indicators of the first monitoring item;

[0008] Randomly generate several monitoring strategies and establish a mapping between the monitoring strategies and the corresponding monitoring data as training samples for the neural network;

[0009] The training samples are input into the neural network, and the decision information is output after the training of the neural network to generate a trained monitoring strategy.

[0010] Furthermore, the first monitoring item and the first monitoring data of the configuration initial monitoring strategy include:

[0011] Set the initial monitoring strategy y, y∈R m ,y=(y1,y2,L,y m ) T , where m is the first monitoring item;

[0012] Configure the first monitoring data x∈R n ,x=(x1,x2,L,x n ) T .

[0013] Furthermore, performing data association on the first monitoring data to generate association rules between first monitoring item monitoring indicators includes:

[0014] The association rule is R1; R1{{C1,M1,S1c},{S1i,N1i},{S1o,N1o}…}; where C1 is the CPU usage; M1 is the memory usage; S1c is the storage usage; S1i is the storage usage during network read operations; N1i is the network read speed; {S1i,N1i} is the association rule between storage and network read speed; S1o is the storage usage during network write operations; N1o is the association rule between network write speed; {S1o,N1o} is the association rule between storage and network write operations.

[0015] Furthermore, the randomly generated monitoring strategies and the mapping between the monitoring strategies and the corresponding monitoring data as training samples of the neural network include: randomly generating P monitoring strategies y=(y 1 ,y 2 ,...y p ), P monitoring strategies correspond to P groups of monitoring data x=(x 1 ,x 2 ,...x p ); so the mapping between monitoring strategy and corresponding monitoring data is (x 1 ,y 1 ), (x 2 ,y 2 ),…,(x P ,y P ).

[0016] Furthermore, the inputting of the training samples into the neural network, outputting decision information after the training of the neural network, and generating a trained monitoring strategy include:

[0017] The training sample is input into the initialized neural network, passes through the input layer, hidden layer and output layer in sequence, and the trained monitoring information is output through the output layer; and the trained monitoring information is compared with the expected output of the monitoring information to calculate the error of the training sample;

[0018] If the error of the training sample cannot meet the monitoring requirements, the weights of each layer of the neural network are modified until the expected output of the control information is satisfied and the decision information is output;

[0019] An optimized monitoring strategy is generated by combining the association rules between the monitoring indicators of the first monitoring item and the decision information.

[0020] Furthermore, the training samples are input into the initialized neural network, sequentially passing through the input layer, hidden layer and output layer, and the monitoring information after training is outputted by the output layer includes:

[0021] If the weight between the input layer and the hidden layer is w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k , then the output of each layer of neurons satisfies:

[0022]

[0023] Where f is the transfer function:

[0024]

[0025] Furthermore, the method further includes: using the optimized monitoring strategy and monitoring threshold to collect monitoring data within a specified time period and re-execute training.

[0026] The present invention also proposes a virtualization system monitoring system, which includes a configuration module, a building module and a training module;

[0027] The configuration module is used to configure the first monitoring item and the first monitoring data of the initial monitoring strategy, and calculate the normal distribution expectation and standard deviation of the first monitoring data; perform data association on the first monitoring data to generate association rules between the monitoring indicators of the first monitoring item;

[0028] The establishment module is used to randomly generate a number of monitoring strategies and establish a mapping between the monitoring strategies and the corresponding monitoring data as training samples for the neural network;

[0029] The training module is used to input the training samples into the neural network, output decision information after the training of the neural network, and generate a trained monitoring strategy.

[0030] The present invention also provides a device comprising:

[0031] memory for storing computer programs;

[0032] A processor is configured to implement the method steps described when executing the computer program.

[0033] The present invention also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method steps described above are implemented.

[0034] The effects provided in the summary of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0035] The present invention proposes a virtualization system monitoring method, system, device and storage medium. The method includes configuring a first monitoring item and first monitoring data of an initial monitoring strategy, and calculating the normal distribution expectation and standard deviation of the first monitoring data; performing data association on the first monitoring data to generate association rules between the monitoring indicators of the first monitoring item; randomly generating a number of monitoring strategies, establishing a mapping between the monitoring strategies and the corresponding monitoring data as training samples for a neural network; inputting the training samples into the neural network, outputting decision information after training of the neural network, and generating a trained monitoring strategy. Based on a virtualization system monitoring method, a virtualization system monitoring system, device and storage medium are also proposed. The present invention obtains original monitoring data or ignition data through traditional monitoring, adopts big data technology to mine association rules of monitoring item indicator data, performs decision analysis on the monitoring strategy through a neural network, updates the monitoring threshold through normal distribution, automatically generates and updates the monitoring strategy, and continuously updates and evolves it. The present invention helps users discover the correlation between monitoring items based on the application of big data, automatically formulates monitoring strategies according to the actual situation of the system, sets thresholds, helps users automatically generate monitoring strategies according to system operation needs, reduces resource consumption of non-correlated monitoring items, and solves the differences in system monitoring strategies caused by different implementation or operation and maintenance personnel capabilities. It uses real-time monitoring data and monitoring needs to objectively and automatically update monitoring strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] like Figure 1 This is a flow chart of a virtualization system monitoring method according to embodiment 1 of the present invention;

[0037] like Figure 2 This is a schematic diagram of the neural network structure of Example 1 of the present invention;

[0038] like Figure 3 This is a schematic diagram of a virtualization system monitoring system according to embodiment 2 of the present invention. DETAILED DESCRIPTION

[0039] In order to clearly illustrate the technical features of this solution, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings. The disclosure below provides many different embodiments or examples for realizing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below. In addition, the present invention may repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplicity and clarity and does not itself indicate the relationship between the various embodiments and / or settings discussed. It should be noted that the components illustrated in the accompanying drawings are not necessarily drawn to scale. The present invention omits descriptions of well-known components and processing technologies and processes to avoid unnecessary limitations on the present invention.

[0040] Example 1

[0041] Embodiment 1 of the present invention proposes a virtualization system monitoring method, which includes manual setting of monitoring strategies, dynamic collection of monitoring data, big data processing, neural network training, and automated generation of monitoring strategies. Manual setting is used to initialize the monitoring strategy for the virtualization system and calculate the normal distribution expectation and standard deviation of the monitoring indicator items; the virtualization system collects monitoring data according to the monitoring strategy; big data technology is used to process and obtain association rules for automated formulation of monitoring strategies; monitoring data is continuously trained and learned by specifying a period of time, generating monitoring strategies; and monitoring strategies are automatically generated through big data and neural networks.

[0042] like Figure 1 A flow chart of a virtualization system monitoring method is given;

[0043] In step S101, the first monitoring item and the first monitoring data of the initial monitoring strategy are configured, and the normal distribution expectation and standard deviation of the first monitoring data are calculated; for big data processing and neural network learning, the virtualization system adopts a traditional monitoring solution at this stage, and the initialization monitoring strategy can be a strategy generated by the experience of operation and maintenance or implementation personnel or other scenarios.

[0044] Configuring the first monitoring item and the first monitoring data of the initial monitoring strategy includes: setting the initial monitoring strategy y, y∈R m ,y=(y1,y2,L,y m ) T , where m is the first monitoring item; configure the first monitoring data x∈R n ,x=(x1,x2,L,x n ) T .

[0045] In step S102, data association is performed on the first monitoring data to generate association rules between the monitoring indicators of the first monitoring item; this serves as an important input for neural network learning and training, and also provides a basis for the subsequent automatic generation of monitoring strategies.

[0046] The association rule is R1; R1{{C1,M1,S1c},{S1i,N1i},{S1o,N1o}…}; C1 is the CPU usage; M1 is the memory usage; S1c is the storage usage; S1i is the storage usage during network read operations; N1i is the network read speed; {S1i,N1i} is the association rule between storage and network read speed; S1o is the storage usage during network write operations; N1o is the association rule between network write speed; {S1o,N1o} is the association rule between storage and network write operations.

[0047] In step S103, a number of monitoring strategies are randomly generated, and a mapping between the monitoring strategies and the corresponding monitoring data is established as training samples for the neural network;

[0048] Randomly generate P monitoring strategies y=(y 1 ,y 2 ,...y p ), P monitoring strategies correspond to P groups of monitoring data x=(x 1 ,x 2 ,...x p ); so the mapping between monitoring strategy and corresponding monitoring data is (x 1 ,y 1 ), (x 2 ,y 2 ),…,(x P ,y P ).

[0049] The training sample is input into the initialized network and passes through the input layer, hidden layer and output layer in sequence. The information of each layer can only be passed to the neurons in the next layer. The output of the output layer is compared with the expected output and the error of the training sample is calculated. If it does not meet the requirements, the second step is carried out. In the second step, the weights of each layer are mainly modified, starting from the output layer and modifying backwards. Then go back to the first step and repeat it many times until the expected goal is achieved and the training ends. Figure 2 Schematic diagram of the neural network structure of Example 1 of the present invention.

[0050] Let the input x∈R n ,x=(x1,x2,L,x n ) T ; The second hidden layer has n1 neurons The final output layer y∈R m , there are m neurons, y=(y1,y2,L,y m ) T , such as the weight between the input layer and the hidden layer is w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w i 'k , the threshold is θ k ', then the output of each layer of neurons satisfies:

[0051]

[0052] Where f is the transfer function:

[0053]

[0054] The training samples are trained repeatedly, and the training is stopped when the overall variance is less than the expected value. The weight between the input layer and the hidden layer is w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k .

[0055] In step S104, the training samples are input into the neural network, and the decision information is output after the training of the neural network to generate the trained monitoring strategy. p+1 Collect monitoring data and use the weight between the input layer and the hidden layer as w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k , substitute into the formula in step 3) to obtain y p +1 New monitoring strategy, update system monitoring strategy.

[0056] In step S105, monitoring data is collected through the new monitoring strategy, and steps S102-S104 are repeated to continue training the neural network, continuously improve and update the recognition of the neural network, and update the system monitoring strategy to realize intelligent virtualization system monitoring based on big data.

[0057] This invention generates monitoring strategies based on big data and neural networks, effectively addressing the complexity of developing them, improving virtualized system monitoring capabilities, and reducing system operating costs through automation. It helps users automatically generate monitoring strategies based on system operational needs, reducing resource consumption for non-correlated monitoring items, and addressing differences in system monitoring strategies caused by varying implementation and maintenance personnel capabilities. It uses real-time monitoring data and monitoring needs to objectively and automatically update monitoring strategies.

[0058] Example 2

[0059] Based on the virtualization system monitoring method proposed in Example 1 of the present invention, Example 2 of the present invention further proposes a virtualization system monitoring system, such as Figure 3This is a schematic diagram of a virtualization system monitoring system according to embodiment 2 of the present invention, the system including a configuration module, a creation module and a training module;

[0060] The configuration module is used to configure the first monitoring item and the first monitoring data of the initial monitoring strategy, and calculate the normal distribution expectation and standard deviation of the first monitoring data; perform data association on the first monitoring data to generate association rules between the monitoring indicators of the first monitoring item;

[0061] Establish a module for randomly generating several monitoring strategies and establish a mapping between the monitoring strategies and the corresponding monitoring data as training samples for the neural network;

[0062] The training module is used to input the training samples into the neural network, output decision information after the training of the neural network, and generate a trained monitoring strategy based on the decision information.

[0063] The process of configuring the module is as follows: configuring the first monitoring item and the first monitoring data of the initial monitoring strategy includes: setting the initial monitoring strategy y, y∈R m ,y=(y1,y2,L,y m ) T , where m is the first monitoring item; configure the first monitoring data x∈R n ,x=(x1,x2,L,x n ) T .

[0064] The association rule is R1; R1{{C1,M1,S1c},{S1i,N1i},{S1o,N1o}…}; C1 is the CPU usage; M1 is the memory usage; S1c is the storage usage; S1i is the storage usage during network read operations; N1i is the network read speed; {S1i,N1i} is the association rule between storage and network read speed; S1o is the storage usage during network write operations; N1o is the association rule between network write speed; {S1o,N1o} is the association rule between storage and network write operations.

[0065] The process of establishing the module implementation is as follows: randomly generate P monitoring strategies y=(y 1 ,y 2 ,...y p ), P monitoring strategies correspond to P groups of monitoring data x=(x 1 ,x 2 ,...x p ); so the mapping between monitoring strategy and corresponding monitoring data is (x 1 ,y 1 ), (x 2 ,y 2 ),…,(x P ,y P ).

[0066] The configuration module is implemented as follows: the training sample is input into the initialized neural network, passes through the input layer, hidden layer and output layer in sequence, and outputs the trained monitoring information through the output layer; and the trained monitoring information is compared with the expected output of the monitoring information to calculate the error of the training sample;

[0067] If the error of the training sample cannot meet the monitoring requirements, the weights of each layer of the neural network are modified until the expected output of the control information is satisfied and the decision information is output;

[0068] An optimized monitoring strategy is generated by combining the association rules between the monitoring indicators of the first monitoring item and the decision information.

[0069] If the weight between the input layer and the hidden layer is w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k , then the output of each layer of neurons satisfies:

[0070]

[0071] Where f is the transfer function:

[0072]

[0073] Through the new monitoring strategy, monitoring data is collected, the neural network is continued to be trained, the recognition of the neural network is continuously improved and updated, and the system monitoring strategy is updated to realize intelligent virtualization system monitoring based on big data.

[0074] Example 3

[0075] The present invention also provides a device comprising:

[0076] memory for storing computer programs;

[0077] The processor is used to implement the following method steps when executing the computer program:

[0078] This method includes manual setting of monitoring strategies, dynamic collection of monitoring data, big data processing, neural network training, and automated generation of monitoring strategies. Manually setting up a virtualized system to initialize the monitoring strategy and calculate the normal distribution expectation and standard deviation of the monitoring indicators; the virtualized system collects monitoring data according to the monitoring strategy; big data technology is used to process and obtain association rules for automated monitoring strategy formulation; monitoring data from a specified period of time is continuously trained and learned to generate monitoring strategies; and monitoring strategies are automatically generated through big data and neural networks.

[0079] like Figure 1 A flow chart of a virtualization system monitoring method is given;

[0080] In step S101, the first monitoring item and the first monitoring data of the initial monitoring strategy are configured, and the normal distribution expectation and standard deviation of the first monitoring data are calculated; for big data processing and neural network learning, the virtualization system adopts a traditional monitoring solution at this stage, and the initialization monitoring strategy can be a strategy generated by the experience of operation and maintenance or implementation personnel or other scenarios.

[0081] Configuring the first monitoring item and the first monitoring data of the initial monitoring strategy includes: setting the initial monitoring strategy y, y∈R m ,y=(y1,y2,L,y m ) T , where m is the first monitoring item; configure the first monitoring data x∈R n ,x=(x1,x2,L,x n ) T .

[0082] In step S102, data association is performed on the first monitoring data to generate association rules between the monitoring indicators of the first monitoring item; this serves as an important input for neural network learning and training, and also provides a basis for the subsequent automatic generation of monitoring strategies.

[0083] The association rule is R1; R1{{C1,M1,S1c},{S1i,N1i},{S1o,N1o}…}; C1 is the CPU usage; M1 is the memory usage; S1c is the storage usage; S1i is the storage usage during network read operations; N1i is the network read speed; {S1i,N1i} is the association rule between storage and network read speed; S1o is the storage usage during network write operations; N1o is the association rule between network write speed; {S1o,N1o} is the association rule between storage and network write operations.

[0084] In step S103, a number of monitoring strategies are randomly generated, and a mapping between the monitoring strategies and the corresponding monitoring data is established as training samples for the neural network;

[0085] Randomly generate P monitoring strategies y=(y 1 ,y 2 ,...y p ), P monitoring strategies correspond to P groups of monitoring data x=(x 1 ,x 2 ,...x p ); so the mapping between monitoring strategy and corresponding monitoring data is (x 1 ,y 1 ), (x 2 ,y 2 ),…,(x P ,yP ).

[0086] The training sample is input into the initialized network and passes through the input layer, hidden layer and output layer in sequence. The information of each layer can only be passed to the neurons in the next layer. The output of the output layer is compared with the expected output and the error of the training sample is calculated. If it does not meet the requirements, the second step is carried out. In the second step, the weights of each layer are mainly modified, starting from the output layer and modifying backwards. Then go back to the first step and repeat it many times until the expected goal is achieved and the training ends. Figure 2 Schematic diagram of the neural network structure of Example 1 of the present invention.

[0087] Let the input x∈R n ,x=(x1,x2,L,x n ) T ; The second hidden layer has n1 neurons The final output layer y∈R m , there are m neurons, y=(y1,y2,L,y m ) T , such as the weight between the input layer and the hidden layer is w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k , then the output of each layer of neurons satisfies:

[0088]

[0089] Where f is the transfer function:

[0090]

[0091] The training samples are trained repeatedly, and the training is stopped when the overall variance is less than the expected value. The weight between the input layer and the hidden layer is w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k .

[0092] In step S104, the training samples are input into the neural network, and the decision information is output after the training of the neural network to generate the trained monitoring strategy. p+1 Collect monitoring data and use the weight between the input layer and the hidden layer as w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k , substitute into the formula in step 3) to obtain y p+1 New monitoring strategy, update system monitoring strategy.

[0093] In step S105, monitoring data is collected through the new monitoring strategy, and steps S102-S104 are repeated to continue training the neural network, continuously improve and update the recognition of the neural network, and update the system monitoring strategy to realize intelligent virtualization system monitoring based on big data.

[0094] This invention generates monitoring strategies based on big data and neural networks, effectively addressing the complexity of developing them, improving virtualized system monitoring capabilities, and reducing system operating costs through automation. It helps users automatically generate monitoring strategies based on system operational needs, reducing resource consumption for non-correlated monitoring items, and addressing differences in system monitoring strategies caused by varying implementation and maintenance personnel capabilities. It uses real-time monitoring data and monitoring needs to objectively and automatically update monitoring strategies.

[0095] It should be noted that the technical solution of the present invention also provides an electronic device, including: a communication interface capable of exchanging information with other devices such as network devices; a processor connected to the communication interface to realize information exchange with other devices, and used to execute a virtualization system monitoring method provided by one or more of the above technical solutions when running a computer program, and the computer program is stored on the memory. Of course, in actual application, the various components in the electronic device are coupled together through a bus system. It can be understood that the bus system is used to realize connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus and a status signal bus. The memory in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of these data include: any computer program for operating on the electronic device. It can be understood that the memory can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface storage, optical disk, or compact disc read-only memory (CD-ROM); magnetic surface storage can be magnetic disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache.By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronized dynamic random access memory (SLDRAM), direct RAM bus random access memory (DRRAM). The memory described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memory. The method disclosed in the above embodiments of the present application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The above-mentioned processor can be a general-purpose processor, a DSP (Digital Signal Processing, i.e., a chip capable of implementing digital signal processing technology), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory. The processor reads the program in the memory and completes the steps of the above method in combination with its hardware. When the processor executes the program, the corresponding processes in the various methods of the embodiments of the present application are implemented. For the sake of brevity, they are not described here.

[0096] Example 4

[0097] The present invention further provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the following method steps are implemented:

[0098] This method includes manual setting of monitoring strategies, dynamic collection of monitoring data, big data processing, neural network training, and automated generation of monitoring strategies. Manually setting up a virtualized system to initialize the monitoring strategy and calculate the normal distribution expectation and standard deviation of the monitoring indicators; the virtualized system collects monitoring data according to the monitoring strategy; big data technology is used to process and obtain association rules for automated monitoring strategy formulation; monitoring data from a specified period of time is continuously trained and learned to generate monitoring strategies; and monitoring strategies are automatically generated through big data and neural networks.

[0099] like Figure 1 A flow chart of a virtualization system monitoring method is given;

[0100] In step S101, the first monitoring item and the first monitoring data of the initial monitoring strategy are configured, and the normal distribution expectation and standard deviation of the first monitoring data are calculated; for big data processing and neural network learning, the virtualization system adopts a traditional monitoring solution at this stage, and the initialization monitoring strategy can be a strategy generated by the experience of operation and maintenance or implementation personnel or other scenarios.

[0101] Configuring the first monitoring item and the first monitoring data of the initial monitoring strategy includes: setting the initial monitoring strategy y, y∈R m ,y=(y1,y2,L,y m ) T , where m is the first monitoring item; configure the first monitoring data x∈R n ,x=(x1,x2,L,x n ) T .

[0102] In step S102, data association is performed on the first monitoring data to generate association rules between the monitoring indicators of the first monitoring item; this serves as an important input for neural network learning and training, and also provides a basis for the subsequent automatic generation of monitoring strategies.

[0103] The association rule is R1; R1{{C1,M1,S1c},{S1i,N1i},{S1o,N1o}…}; C1 is the CPU usage; M1 is the memory usage; S1c is the storage usage; S1i is the storage usage during network read operations; N1i is the network read speed; {S1i,N1i} is the association rule between storage and network read speed; S1o is the storage usage during network write operations; N1o is the association rule between network write speed; {S1o,N1o} is the association rule between storage and network write operations.

[0104] In step S103, a number of monitoring strategies are randomly generated, and a mapping between the monitoring strategies and the corresponding monitoring data is established as training samples for the neural network;

[0105] Randomly generate P monitoring strategies y=(y 1 ,y 2 ,...y p ), P monitoring strategies correspond to P groups of monitoring data x=(x 1 ,x 2 ,...x p ); so the mapping between monitoring strategy and corresponding monitoring data is (x 1 ,y 1 ), (x 2 ,y 2 ),…,(x P ,y P ).

[0106] The training sample is input into the initialized network and passes through the input layer, hidden layer and output layer in sequence. The information of each layer can only be passed to the neurons in the next layer. The output of the output layer is compared with the expected output and the error of the training sample is calculated. If it does not meet the requirements, the second step is carried out. In the second step, the weights of each layer are mainly modified, starting from the output layer and modifying backwards. Then go back to the first step and repeat it many times until the expected goal is achieved and the training ends. Figure 2 Schematic diagram of the neural network structure of Example 1 of the present invention.

[0107] Let the input x∈R n ,x=(x1,x2,L,x n ) T ; The second hidden layer has n1 neurons The final output layer y∈R m , there are m neurons, y=(y1,y2,L,y m ) T , such as the weight between the input layer and the hidden layer is w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k , then the output of each layer of neurons satisfies:

[0108]

[0109] Where f is the transfer function:

[0110]

[0111] The training samples are trained repeatedly, and the training is stopped when the overall variance is less than the expected value. The weight between the input layer and the hidden layer is wik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k .

[0112] In step S104, the training samples are input into the neural network, and the decision information is output after the training of the neural network to generate the trained monitoring strategy. p+1 Collect monitoring data and use the weight between the input layer and the hidden layer as w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w′ ik , the threshold is θ′ k , substitute into the formula in step 3) to obtain y p +1 New monitoring strategy, update system monitoring strategy.

[0113] In step S105, monitoring data is collected through the new monitoring strategy, and steps S102-S104 are repeated to continue training the neural network, continuously improve and update the recognition of the neural network, and update the system monitoring strategy to realize intelligent virtualization system monitoring based on big data.

[0114] This invention generates monitoring strategies based on big data and neural networks, effectively addressing the complexity of developing them, improving virtualized system monitoring capabilities, and reducing system operating costs through automation. It helps users automatically generate monitoring strategies based on system operational needs, reducing resource consumption for non-correlated monitoring items, and addressing differences in system monitoring strategies caused by varying implementation and maintenance personnel capabilities. It uses real-time monitoring data and monitoring needs to objectively and automatically update monitoring strategies.

[0115] The present application also provides a storage medium, namely, a computer storage medium, specifically a computer-readable storage medium, such as a memory storing a computer program, which can be executed by a processor to perform the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface mount storage, optical disk, or CD-ROM.

[0116] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments can be implemented by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks. Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. And the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0117] For the description of the relevant parts of the processing device and storage medium for virtualization system monitoring provided in the embodiment of the present application, please refer to the detailed description of the corresponding parts in the virtualization system monitoring method provided in Example 1 of the present application, and will not be repeated here.

[0118] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements are inherent to the elements. In the absence of further restrictions, the elements limited by the statement "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device comprising the elements. In addition, the above-mentioned technical solutions provided in the embodiments of the present application are not described in detail in accordance with the corresponding technical solutions in the prior art to achieve the same principle, so as to avoid excessive elaboration.

[0119] Although the above description is of specific embodiments of the present invention in conjunction with the accompanying drawings, it does not limit the scope of protection of the present invention. For those skilled in the art, other different forms of modifications or variations can be made based on the above description. It is not necessary and impossible to list all embodiments here. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without expending creative effort are still within the scope of protection of the present invention.

Claims

1. A virtualization system monitoring method, characterized in that: The following steps are involved: Configure a first monitoring item and first monitoring data of the initial monitoring strategy, and calculate the normal distribution expectation and standard deviation of the first monitoring data; Performing data association on the first monitoring data to generate association rules between the first monitoring item and monitoring indicators; the first monitoring item and the first monitoring data of the configuration initial monitoring strategy include: setting the initial monitoring strategy y, y∈R m ,y=(y1,y2,…,y m ) T , where m is the first monitoring item; configure the first monitoring data x∈R n ,x=(x1,x2,…,x n ) T ; The first monitoring data is associated with data to generate association rules between monitoring indicators of the first monitoring item, including: the association rule is R1; R1{{C1, M1, S1c}, {S1i, N1i}, {S1o, N1o}…}; wherein C1 is CPU usage; M1 is memory usage; S1c is storage usage; S1i is storage usage during network read operations; N1i is network read speed; {S1i, N1i} is an association rule for storage and network read speed; S1o is storage usage during network write operations; N1o is an association rule for network write speed; {S1o, N1o} is an association rule for storage and network write operations; Randomly generate several monitoring strategies, establish the mapping between monitoring strategies and corresponding monitoring data as training samples of the neural network, including: randomly generate P monitoring strategies y = (y 1 ,y 2 ,...y p ), P monitoring strategies correspond to P groups of monitoring data x=(x 1 ,x 2 ,...x p ); so the mapping between monitoring strategy and corresponding monitoring data is (x 1 ,y 1 ), (x 2 ,y 2 ),…,(x P ,y P ); Randomly generate several monitoring strategies and establish a mapping between the monitoring strategies and the corresponding monitoring data as training samples for the neural network; The training samples are input into a neural network, decision information is output after training of the neural network, and a trained monitoring strategy is generated according to the decision information.

2. A virtualization system monitoring method according to claim 1, characterized in that: Inputting the training samples into a neural network, outputting decision information after training of the neural network, and generating a trained monitoring strategy based on the decision information include: The training sample is input into the initialized neural network, passes through the input layer, hidden layer and output layer in sequence, and the trained monitoring information is output through the output layer; and the trained monitoring information is compared with the expected output of the monitoring information to calculate the error of the training sample; If the error of the training sample cannot meet the monitoring requirements, the weights of each layer of the neural network are modified until the expected output of the control information is satisfied and the decision information is output; An optimized monitoring strategy is generated by combining the association rules between the monitoring indicators of the first monitoring item and the decision information.

3. A virtualization system monitoring method according to claim 2, characterized in that: The training samples are input into the initialized neural network, sequentially passing through the input layer, hidden layer and output layer, and the monitoring information after training is outputted through the output layer includes: If the weight between the input layer and the hidden layer is w ik , the threshold is θ j , the weight between the hidden layer and the output layer is w' ik , the threshold is θ' k , then the output of each layer of neurons satisfies: Where f is the transfer function:

4. A virtualization system monitoring method according to claim 3, characterized in that: The method further includes: using the optimized monitoring strategy and monitoring threshold to collect monitoring data within a specified time period and re-execute training.

5. A virtualization system monitoring system, configured to execute a virtualization system monitoring method according to any one of claims 1 to 4, characterized in that: Includes configuration module, establishment module and training module; The configuration module is used to configure the first monitoring item and the first monitoring data of the initial monitoring strategy, and calculate the normal distribution expectation and standard deviation of the first monitoring data; perform data association on the first monitoring data to generate association rules between the monitoring indicators of the first monitoring item; The establishment module is used to randomly generate a number of monitoring strategies and establish a mapping between the monitoring strategies and the corresponding monitoring data as training samples for the neural network; The training module is used to input the training samples into the neural network, output decision information after the training of the neural network, and generate a trained monitoring strategy based on the decision information.

6. A device, characterized in that include: memory for storing computer programs; A processor, configured to implement the method steps according to any one of claims 1 to 4 when executing the computer program.

7. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements the method steps according to any one of claims 1 to 4.

Images