Cloud service failure pre-warning method and device
By combining generative adversarial network (GAN) models and convolutional long short-term memory (LSTM) networks, an indicator data prediction model is constructed, which solves the problem of low accuracy in indicator data prediction in cloud service fault early warning and realizes accurate fault early warning for cloud services.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA TELECOM CORP LTD
- Filing Date
- 2023-07-14
- Publication Date
- 2026-07-03
AI Technical Summary
Existing technologies have low accuracy in predicting indicator data for cloud service fault early warning, making it difficult to provide effective reference for fault early warning.
By employing a generative adversarial network model, combined with a convolutional long short-term memory network and a codec, and through information distribution structure analysis and sequence continuity analysis, an indicator data prediction model is constructed. The model parameters are adjusted using training and validation datasets to achieve early warning of cloud service failures.
It improves the accuracy of indicator data prediction, ensures the reliability and precision of prediction results, can issue abnormal alarm information in a timely manner, and enhances the operational reliability of cloud services.
Smart Images
Figure CN116723078B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of information processing technology, and more specifically, to a method and apparatus for early warning of cloud service failures. Background Technology
[0002] With the continuous development of internet and communication technologies, the scale of system informatization and cloud migration of applications in facilities, management, and business based on the internet is growing larger and larger. Correspondingly, the difficulty of maintaining and supervising cloud resources and equipment is also increasing. By monitoring and analyzing historical indicator data to achieve fault early warning, the reliability of system operation can be effectively improved, transforming the work mode of operation and maintenance personnel from passively responding to temporary emergencies to proactive planning, thus ensuring the quality of cloud services and user experience.
[0003] The core of fault early warning is the accurate prediction of time series index data. Statistical prediction methods in related technologies usually rely on a single model or a combination of multiple models to model and predict index data, such as: (1) using high-order Markov chains to predict rainfall. This method identifies outliers and divides states, performs multi-step weighted calculation of transition probabilities for the divided states, and then predicts the next state; (2) using ordered clustering transition matrices to predict monitoring cloud maps. This method calculates the state transition matrix by ordered distance of cloud map data and introduces a membership function to consider the correlation between states, thus achieving the prediction of monitoring data; (3) using support vector regression to predict computer performance. This method uses optimal regularized soft support vector regression to improve the stability of the model in multidimensional situations. The above methods can be summarized as using polynomial functions to fit the changing trend of real index data. Therefore, they perform well in scenarios where the time series changes are linearly stable and the feature dimensions are low. However, in cloud service fault prediction scenarios, taking the prediction of the running status of PAAS (Platform As A Service) components as an example, the data is affected by a variety of complex factors such as CPU running status, memory and disk usage, and network traffic reception rate. The significance and uncertainty of these factors cause the above model to deviate significantly from the actual situation when predicting data in this scenario.
[0004] In recent years, with the development of deep learning technology, technologies such as LSTM (Long Short-Term Memory) and GAN (Generative Adversarial Networks) have provided more ideas for spatiotemporal sequence prediction problems. Some general deep learning time series data prediction models have achieved advanced performance in their respective application scenarios. However, they cannot be directly used for indicator data prediction tasks in fault warning scenarios. This is because the data change patterns in fault warning scenarios are complex and strongly correlated with multi-dimensional features. Ignoring the prior features of various dimensions related to the data to be predicted will make it difficult for the model to fully capture the temporal information, which increases the uncertainty of the model's prediction direction. Existing models will easily combine multiple possible results to give fuzzy predictions. In addition, common loss functions focus on the corresponding pixel differences between the generated results and the real data. However, the indicator curves in warning scenarios usually account for a small proportion of the overall data. Considering only the differences between pixel values will not constrain the model to focus on the details of indicator change trends, resulting in difficult model training and poor generalization ability.
[0005] There is currently no effective solution to the above problems. Summary of the Invention
[0006] This application provides a cloud service fault early warning method and apparatus to at least solve the technical problem in related technologies where the accuracy of predicting cloud service indicator data is low, making it difficult to provide a reference for fault early warning.
[0007] According to one aspect of the embodiments of this application, a cloud service fault early warning method is provided, comprising: acquiring a first indicator data sequence of a first dimension and a second indicator data sequence of a preset number of second dimensions in a cloud service, wherein the second dimension is associated with the first dimension; performing information distribution structure analysis and sequence continuity analysis on the first indicator data sequence and the second indicator data sequence using a pre-trained indicator data prediction model to obtain a first prediction result, wherein the first prediction result is the predicted next indicator data adjacent to the first indicator data sequence; and determining that the cloud service is abnormal and issuing an abnormal alarm message when the first prediction result does not meet the preset indicator data threshold range.
[0008] Optionally, the training process of the indicator data prediction model includes: constructing an initial prediction model, wherein the initial prediction model is a generative adversarial network model; obtaining a sample dataset and dividing the sample dataset into a training dataset, a validation dataset, and a test dataset according to a preset ratio; iteratively training the initial prediction model using the training dataset and adjusting the hyperparameters of the initial prediction model using the validation dataset to obtain the indicator data prediction model; and testing the indicator data prediction model using the test dataset.
[0009] Optionally, constructing an initial prediction model includes: constructing a generator, wherein the generator includes a preset number of convolutional long short-term memory networks and a codec, the codec including multiple downsampling convolutional layers, multiple deep residual blocks, and multiple upsampling convolutional layers; constructing a discriminator, wherein the discriminator includes a spatial feature discriminator for information distribution structure analysis and a temporal feature discriminator for sequence continuity analysis, each discriminator including multiple downsampling modules, each downsampling module including a downsampling convolutional layer and a linear unit with leakage correction; constructing an initial prediction model based on the generator and discriminator, and determining the generator parameters and discriminator parameters based on the size of the data of the index to be predicted, wherein the generator parameters include the size of the downsampling layer convolutional kernel, the size of the upsampling layer convolutional kernel, and the size of the residual block, and the discriminator parameters include the size of the downsampling layer convolutional kernel and the parameters of the linear unit with leakage correction.
[0010] Optionally, obtaining a sample dataset includes: obtaining multiple sets of sample data from historical indicator data of cloud services to obtain a sample dataset, wherein each set of sample data includes: a third indicator data sequence of the third dimension, a fourth indicator data sequence of a preset number of fourth dimensions, and annotation information corresponding to the third indicator data sequence. The fourth dimension is associated with the third dimension, and the annotation information is used to characterize the next indicator data that is immediately adjacent to the third indicator data sequence.
[0011] Optionally, the initial prediction model is iteratively trained using the training dataset, including: for each training sample in the training dataset, using a preset number of convolutional long short-term memory networks to extract features from a preset number of fourth indicator data sequences in the training sample data, obtaining a preset number of associated features; stacking the preset number of associated features with the third indicator data sequence in the training sample data along the channel dimension to obtain an input vector; inputting the input vector into a codec to obtain a second prediction result output by the codec, wherein the second prediction result is the predicted next indicator data immediately adjacent to the third indicator data sequence; and combining the annotation information in the training sample data with the second prediction result. The test results are input into the spatial feature discriminator to obtain the first and second classification results output by the spatial feature discriminator. The annotation information in the training sample data is combined with the third indicator data sequence to form the fifth indicator data sequence, and the second prediction result is combined with the third indicator data sequence to form the sixth indicator data sequence. The fifth and sixth indicator data sequences are input into the temporal feature discriminator to obtain the third and fourth classification results output by the temporal feature discriminator. A target loss function is constructed based on the first, second, third, and fourth classification results. During iterative training, the model parameters of the initial prediction model are adjusted according to the target loss function.
[0012] Optionally, a target loss function is constructed based on the first classification result, the second classification result, the third classification result, and the fourth classification result, including: constructing a spatial distribution loss function based on the first classification result and the second classification result; constructing a sequence loss function based on the third classification result and the fourth classification result; and weighted summing of the spatial distribution loss function and the sequence loss function to obtain the target loss function.
[0013] Optionally, the indicator data prediction model is tested using a test dataset, including: for each test sample data in the test dataset, inputting the third indicator data sequence and the fourth indicator data sequence from the test sample data into the indicator data prediction model to obtain the third prediction result output by the indicator data prediction model; determining test evaluation data based on the third prediction result and the annotation information in the test sample data, wherein the type of test evaluation data includes at least one of the following: mean squared error, structural similarity, peak signal-to-noise ratio; and determining that the indicator data prediction model is qualified when the test evaluation data is greater than a preset threshold.
[0014] According to another aspect of the embodiments of this application, a cloud service fault early warning device is also provided, comprising: an acquisition module, configured to acquire a first indicator data sequence of a first dimension and a second indicator data sequence of a preset number of second dimensions in a cloud service, wherein the second dimension is associated with the first dimension; an analysis module, configured to perform information distribution structure analysis and sequence continuity analysis on the first indicator data sequence and the second indicator data sequence using a pre-trained indicator data prediction model to obtain a first prediction result, wherein the first prediction result is the predicted next indicator data adjacent to the first indicator data sequence; and an alarm module, configured to determine that there is an anomaly in the cloud service and issue an anomaly alarm message when the first prediction result does not meet the preset indicator data threshold range.
[0015] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-described cloud service fault early warning method by running the computer program.
[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described cloud service fault early warning method through the computer program.
[0017] In this embodiment, firstly, a first indicator data sequence of a first dimension and a preset number of second indicator data sequences of a second dimension associated with the first dimension are obtained from the cloud service. Then, a pre-trained indicator data prediction model is used to perform information distribution structure analysis and sequence continuity analysis on the first and second indicator data sequences to obtain a first prediction result, which is the predicted next indicator data immediately adjacent to the first indicator data sequence. When the first prediction result does not meet a preset indicator data threshold range, it is determined that there is an anomaly in the cloud service, and an anomaly alarm is issued. By comprehensively analyzing the indicator data sequence to be predicted and the indicator data sequences of its related dimensions, the accuracy of the prediction direction can be ensured. Through information distribution structure analysis and sequence continuity analysis, the inherent relationship between the spatial distribution characteristics and temporal change patterns of the data can be uncovered, ensuring the reliability of the prediction result. This allows for accurate fault early warning for the cloud service. This application effectively solves the technical problem in related technologies where the accuracy of indicator data prediction for cloud services is low, making it difficult to provide a reference for fault early warning. Attached Figure Description
[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0019] Figure 1 This is a schematic diagram of the structure of an optional computer terminal according to an embodiment of this application;
[0020] Figure 2 This is a flowchart illustrating an optional cloud service fault early warning method according to an embodiment of this application;
[0021] Figure 3 This is a schematic diagram of an optional indicator data prediction model training process according to an embodiment of this application;
[0022] Figure 4 This is a schematic diagram of the structure of an optional initial prediction model according to an embodiment of this application;
[0023] Figure 5 This is a schematic diagram of an optional initial prediction model training process according to an embodiment of this application;
[0024] Figure 6 This is a schematic diagram of an optional cloud service fault early warning device according to an embodiment of this application. Detailed Implementation
[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0026] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0027] Example 1
[0028] According to an embodiment of this application, a cloud service fault early warning method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0029] The method embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a cloud service fault early warning method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.
[0030] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).
[0031] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the cloud service fault early warning method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned application vulnerability detection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0032] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.
[0033] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).
[0034] Under the above operating environment, this application embodiment provides a cloud service fault early warning method, such as... Figure 2 As shown, the method includes the following steps:
[0035] Step S202: Obtain the first indicator data sequence of the first dimension and the second indicator data sequence of the second dimension of the preset number of groups in the cloud service, wherein the second dimension is associated with the first dimension;
[0036] Step S204: Use the pre-trained indicator data prediction model to perform information distribution structure analysis and sequence continuity analysis on the first indicator data sequence and the second indicator data sequence to obtain the first prediction result, wherein the first prediction result is the predicted next indicator data that is immediately adjacent to the first indicator data sequence.
[0037] Step S206: When the first prediction result does not meet the preset indicator data threshold range, it is determined that there is an anomaly in the cloud service and an anomaly alarm message is issued.
[0038] The first indicator data sequence of the first dimension can be a monitoring indicator data sequence of a component in the PaaS layer of cloud services, while the second indicator data sequence of the second dimension can be a monitoring indicator data sequence of the IaaS (Infrastructure As A Service) layer related to the operation of the component, such as CPU utilization, memory and disk utilization, network traffic reception rate and other indicator data sequences.
[0039] By comprehensively analyzing the first indicator data sequence to be predicted and the second indicator data sequence of its related dimensions through the indicator data prediction model, the accuracy of the prediction direction can be guaranteed. Through information distribution structure analysis and sequence continuity analysis, the intrinsic relationship between the spatial distribution characteristics and temporal change patterns of the indicator data can be discovered, ensuring the reliability of the prediction results, thereby accurately realizing fault early warning for cloud services.
[0040] The following section explains each step of the cloud service fault early warning method in conjunction with the specific implementation process.
[0041] As an optional implementation method, such as Figure 3 As shown, the indicator data prediction model can be trained by following these steps:
[0042] S302, Construct the initial prediction model, which is a generative adversarial network model.
[0043] The design of the generative adversarial network model is based on game theory. It has at least one pair of generators and discriminators. The generator's goal is to output something that can "deceive" the discriminator, while the discriminator's task is to comprehensively examine the generator's output from both local texture details and global structural features, thus constraining the generator's final output to approximate the real data distribution.
[0044] Optionally, the generator constructed in this application embodiment includes: a preset number of Convolutional Long Short-Term Memory (Conv-LSTM) networks and a codec. The preset number can be adjusted according to the needs. The codec includes: multiple downsampling convolutional layers, multiple depth residual blocks, and multiple upsampling convolutional layers.
[0045] It should be noted that LSTM networks are designed to predict reasonable future outcomes across multiple time spans. Their carefully designed memory and forgetting units control the recording and deletion of correlated information. Correlated information is only computed in small amounts by relevant modules during the stage where the model examines the correlation between input and historical information, effectively avoiding gradient vanishing and gradient explosion when the prediction model faces nonlinear relationships across multiple time spans. Conv-LSTM networks improve upon this, possessing the ability to compute temporal and spatial correlations in the input sequence, allowing the model to capture temporal information between sequences without compromising spatial information.
[0046] The constructed discriminators include a spatial feature discriminator for information distribution structure analysis and a temporal feature discriminator for sequence continuity analysis. Each discriminator includes multiple downsampling modules, each containing a downsampling convolutional layer and a linear unit with leakage correction. The spatial feature discriminator compares the information structure distribution of the predicted results with that of the actual index data, while the temporal feature discriminator compares the temporal continuity of the sequence containing the predicted results with that of the actual index data sequence. Both discriminators constrain the generator to produce more accurate predictions based on the spatial distribution of the predicted data and the temporal continuity between the sequences containing the predicted results, respectively.
[0047] Figure 4 A schematic diagram of an optional initial prediction model is shown, in which the generator includes two convolutional long short-term memory networks, Conv-LSTM1 and Conv-LSTM2, whose outputs are concatenated to a codec consisting of 3 downsampling convolutional layers, 9 deep residual blocks (ResBlock), and 3 upsampling convolutional layers; the spatial feature discriminator and the temporal feature discriminator each consist of 4 downsampling modules, each of which includes a downsampling convolutional layer and a linear unit (Leaky ReLU) with leakage correction.
[0048] After constructing the initial prediction model based on the generator and discriminator, the generator parameters and discriminator parameters can be determined according to the size of the data of the index to be predicted. The generator parameters include: the size of the downsampling layer convolution kernel, the size of the upsampling layer convolution kernel, and the size of the residual block. The discriminator parameters include: the size of the downsampling layer convolution kernel and the linear unit parameters with leakage correction.
[0049] S304. Obtain the sample dataset and divide it into training dataset, validation dataset, and test dataset according to a preset ratio.
[0050] Optionally, when obtaining the sample dataset, multiple sets of sample data can be obtained from the historical indicator data of the cloud service to obtain the sample dataset. Each set of sample data includes: a third indicator data sequence of the third dimension, a fourth indicator data sequence of a preset number of fourth dimensions, and annotation information corresponding to the third indicator data sequence. The fourth dimension is associated with the third dimension, and the annotation information is used to characterize the next indicator data that is immediately adjacent to the third indicator data sequence.
[0051] The third dimension's third indicator data sequence can be a monitoring indicator data sequence of a component in the PaaS layer of cloud services, while the fourth dimension's fourth indicator data sequence can be a monitoring indicator data sequence of the IaaS layer related to the operation of that component, such as CPU utilization, memory and disk utilization, network traffic reception rate, and other indicator data sequences.
[0052] When dividing the training, validation, and test datasets, the preset ratios can be adjusted according to needs. For example, 70% of the data in the sample dataset can be used for training, 25% for testing, and 5% for validation.
[0053] S306. The initial prediction model is iteratively trained using the training dataset, and the hyperparameters of the initial prediction model are adjusted using the validation dataset to obtain the indicator data prediction model.
[0054] As an optional implementation, during iterative training of the initial prediction model: for each training sample in the training dataset, a preset number of convolutional long short-term memory networks can be used to extract features from a preset number of fourth indicator data sequences in the training sample data, obtaining a preset number of associated features; the preset number of associated features are stacked with the third indicator data sequence in the training sample data by channel dimension to obtain an input vector; the input vector is input to a codec to obtain a second prediction result output by the codec, wherein the second prediction result is the predicted next indicator data immediately adjacent to the third indicator data sequence; the annotation information in the training sample data and the second... The prediction results are input into the spatial feature discriminator to obtain the first and second classification results output by the spatial feature discriminator. The annotation information in the training sample data is combined with the third indicator data sequence to form the fifth indicator data sequence, and the second prediction result is combined with the third indicator data sequence to form the sixth indicator data sequence. The fifth and sixth indicator data sequences are input into the temporal feature discriminator to obtain the third and fourth classification results output by the temporal feature discriminator. A target loss function is constructed based on the first, second, third, and fourth classification results. During iterative training, the model parameters of the initial prediction model are adjusted according to the target loss function.
[0055] Optionally, when constructing the target loss function based on the first classification result, the second classification result, the third classification result, and the fourth classification result, a spatial distribution loss function can be constructed based on the first classification result and the second classification result; a sequence loss function can be constructed based on the third classification result and the fourth classification result; and the spatial distribution loss function and the sequence loss function can be weighted and summed to obtain the target loss function.
[0056] Figure 5 A schematic diagram of an optional initial prediction model training process is shown, wherein the third indicator data sequence in the training sample data is {x}. m ,x m+1 ,…x n The two sets of fourth indicator data sequences associated with it are {y}. m ,y m+1 ,…y n} and {z m ,z m+1 ,…z n}, the annotation information is x n+1 .
[0057] First, the two sets of fourth indicator data sequences {y m ,y m+1 ,…y n} and {z m ,z m+1 ,…z nThe extracted association features t1 and t2 are obtained by inputting them into two Conv-LSTM networks respectively; these features are then compared with the third indicator data sequence {x}. m ,x m+1 ,…x n Stacking the channels together yields the input vector {x}. m ,x m+1 ,…x n ∪t1,t2};The input vector {x} m ,x m+1 ,…x n Inputting ∪t1,t2} into the codec G yields the second prediction result G(x). m ,x m+1 ,…x n ∪t1,t2)=x n+1 * ; label information x n+1 Second prediction result x n+1 * Input spatial feature discriminator D respectively p The first classification result D is obtained. p (x n+1 ) and the second classification result D p (x n+1 * ); The annotation information x n+1 With the third indicator data sequence {x m ,x m+1 ,…x n} forms the fifth indicator data sequence {x m :x n+1}, the second prediction result x n+1 * With the third indicator data sequence {x m ,x m+1 ,…x n} forms the sixth indicator data sequence {x m :x n+1 *}, then the fifth indicator data sequence {x m :x n+1} and the sixth indicator data sequence {x m :x n+1 * Input the temporal feature discriminator D respectively t The third classification result D was obtained. t (x m :x n+1 ) and the fourth classification result D t (x m :x n+1 * ).
[0058] Then, based on the first classification result D p (x n+1 ) and the second classification result D p (x n+1 * Construct the spatial distribution loss function:
[0059]
[0060] Based on the third classification result D t (x m :x n+1 ) and the fourth classification result D t (x m :x n+1 * Construct the sequence loss function:
[0061]
[0062] Determine the spatial distribution loss function L respectively frame and sequence loss function L seq The weights are w1 and w2, and the weighted average yields the target loss function as follows:
[0063] L p =w1·L frame +w2·L seq
[0064] After setting the relevant training parameters such as the number of training epochs, the number of samples input to the model per cycle, and the learning rate, iterative training can be performed. During training, the discriminator parameters can be fixed first, allowing the generator's output to deceive the discriminator's standard. Then, the generator parameters can be fixed, and the discriminator can be trained to successfully distinguish between the output data and the real data.
[0065] In the phase where the discriminator parameters are fixed, if the generator's output is distinguished from the real data by the spatial feature discriminator, it indicates that the generator's modeling of the structural distribution of the data to be predicted still has defects. Conversely, if the sequence containing the output is identified by the temporal feature discriminator, it proves that the generator's capture of the continuity features of the predicted sequence is still inaccurate. In the phase where the generator parameters are fixed, if the discriminator cannot distinguish between real data and generated data in terms of spatial distribution characteristics and temporal continuity, it indicates that the discriminator's judgment ability needs improvement. Through repeated training iterations, when the model generator and discriminator reach a Nash equilibrium, it indicates that the model has implicitly modeled the spatial structure and temporal features of the monitored data, and the model training is complete.
[0066] S308, use the test dataset to test the indicator data prediction model.
[0067] To verify the accuracy of the trained indicator data prediction model, the model can be tested using a test dataset in the following manner: For each test sample in the test dataset, the third and fourth indicator data sequences from the test sample are input into the indicator data prediction model to obtain the third prediction result output by the model; based on the third prediction result and the annotation information in the test sample data, test evaluation data is determined, wherein the type of test evaluation data includes at least one of the following: mean squared error (MSE), structural similarity (SSIM), and peak signal-to-noise ratio (PSNR); if the test evaluation data exceeds a preset threshold, the indicator data prediction model is deemed to have passed the test.
[0068] Throughout the training process, the memory and forgetting units of the Conv-LSTM network were used to capture the temporal features of sequences associated with the data to be predicted, effectively reducing the uncertainty of multi-dimensional contextual information in cloud service fault warning scenarios. At the same time, the indicator prediction loss function introduced adversarial learning and temporal consistency training strategies, comparing the model prediction results with the information structure distribution and temporal continuity of the real data, iteratively updating the model parameters, and exploring the intrinsic relationship between the temporal change patterns and spatial distribution characteristics of complex indicator data. The prediction results of the trained model eventually converged to the measured indicator data distribution, solving the problems of difficult model training and poor generalization ability in this scenario.
[0069] In practical applications, the monitoring indicator data sequence of a component in the PaaS layer of the cloud service is obtained as the first indicator data sequence to be predicted, and the monitoring indicator data sequence of the IaaS layer related to the operation of this component is used as the second indicator data sequence. Simply input the first and second indicator data sequences into the trained indicator data prediction model, and the model can predict the next indicator data for the first indicator data sequence as the first prediction result. If the first prediction result does not meet the preset indicator data threshold range, it is determined that there is an anomaly in the cloud service, and an anomaly alarm can be issued. The indicator data threshold range can be set empirically.
[0070] In this embodiment, firstly, a first indicator data sequence of a first dimension and a preset number of second indicator data sequences of a second dimension associated with the first dimension are obtained from the cloud service. Then, a pre-trained indicator data prediction model is used to perform information distribution structure analysis and sequence continuity analysis on the first and second indicator data sequences to obtain a first prediction result, which is the predicted next indicator data immediately adjacent to the first indicator data sequence. When the first prediction result does not meet a preset indicator data threshold range, it is determined that there is an anomaly in the cloud service, and an anomaly alarm is issued. By comprehensively analyzing the indicator data sequence to be predicted and the indicator data sequences of its related dimensions, the accuracy of the prediction direction can be ensured. Through information distribution structure analysis and sequence continuity analysis, the inherent relationship between the spatial distribution characteristics and temporal change patterns of the data can be uncovered, ensuring the reliability of the prediction result. This allows for accurate fault early warning for the cloud service. This application effectively solves the technical problem in related technologies where the accuracy of indicator data prediction for cloud services is low, making it difficult to provide a reference for fault early warning.
[0071] Example 2
[0072] According to an embodiment of this application, a cloud service failure early warning device is also provided for implementing the cloud service failure early warning method in Embodiment 1, such as... Figure 6 As shown, the cloud service fault early warning device includes at least an acquisition module 61, an analysis module 62, and an alarm module 63, wherein:
[0073] The acquisition module 61 is used to acquire the first indicator data sequence of the first dimension and the second indicator data sequence of the second dimension of the preset number of groups in the cloud service, wherein the second dimension is associated with the first dimension.
[0074] The first indicator data sequence of the first dimension can be a monitoring indicator data sequence of a component in the PaaS layer of the cloud service, while the second indicator data sequence of the second dimension can be a monitoring indicator data sequence of the IaaS layer related to the operation of the component, such as CPU utilization, memory and disk utilization, network traffic reception rate and other indicator data sequences.
[0075] Analysis module 62 is used to perform information distribution structure analysis and sequence continuity analysis on the first indicator data sequence and the second indicator data sequence using a pre-trained indicator data prediction model to obtain a first prediction result, wherein the first prediction result is the predicted next indicator data that is immediately adjacent to the first indicator data sequence.
[0076] The alarm module 63 is used to determine that there is an anomaly in the cloud service and issue an anomaly alarm message when the first prediction result does not meet the preset indicator data threshold range.
[0077] Optionally, the cloud service fault early warning device in this application embodiment further includes a model training module, which is used to train the indicator data prediction model in the following manner: constructing an initial prediction model, which is a generative adversarial network model; obtaining a sample dataset and dividing the sample dataset into a training dataset, a validation dataset, and a test dataset according to a preset ratio; iteratively training the initial prediction model using the training dataset and adjusting the hyperparameters of the initial prediction model using the validation dataset to obtain the indicator data prediction model; and testing the indicator data prediction model using the test dataset.
[0078] The design of the generative adversarial network model is based on game theory. It has at least one pair of generators and discriminators. The generator's goal is to output something that can "deceive" the discriminator, while the discriminator's task is to comprehensively examine the generator's output from both local texture details and global structural features, thus constraining the generator's final output to approximate the real data distribution.
[0079] Optionally, the generator built by the model training module includes: a preset number of Convolutional Long Short-Term Memory (Conv-LSTM) networks and a codec. The preset number can be adjusted according to the needs. The codec includes: multiple downsampling convolutional layers, multiple depth residual blocks, and multiple upsampling convolutional layers.
[0080] It should be noted that LSTM networks are designed to predict reasonable future outcomes across multiple time spans. Their carefully designed memory and forgetting units control the recording and deletion of correlated information. Correlated information is only computed in small amounts by relevant modules during the stage where the model examines the correlation between input and historical information, effectively avoiding gradient vanishing and gradient explosion when the prediction model faces nonlinear relationships across multiple time spans. Conv-LSTM networks improve upon this, possessing the ability to compute temporal and spatial correlations in the input sequence, allowing the model to capture temporal information between sequences without compromising spatial information.
[0081] The discriminators constructed by the model training module include: a spatial feature discriminator for information distribution structure analysis and a temporal feature discriminator for sequence continuity analysis. Each discriminator includes multiple downsampling modules, each of which includes a downsampling convolutional layer and a linear unit with leakage correction. The spatial feature discriminator compares the information structure distribution of the predicted results with that of the actual index data, while the temporal feature discriminator compares the temporal continuity of the sequence containing the predicted results with that of the actual index data sequence. Both discriminators constrain the generator to produce more accurate predictions based on the spatial distribution of the predicted data and the temporal continuity between the sequences containing the predicted results, respectively.
[0082] After constructing the initial prediction model based on the generator and discriminator, the model training module can determine the generator parameters and discriminator parameters based on the size of the data of the index to be predicted. The generator parameters include the size of the downsampling layer convolution kernel, the size of the upsampling layer convolution kernel, and the size of the residual block. The discriminator parameters include the size of the downsampling layer convolution kernel and the parameters of the linear unit with leakage correction.
[0083] Optionally, when the model training module obtains the sample dataset, it can obtain multiple sets of sample data from the historical indicator data of the cloud service to obtain the sample dataset. Each set of sample data includes: the third indicator data sequence of the third dimension, the fourth indicator data sequence of the fourth dimension with a preset number of sets, and the annotation information corresponding to the third indicator data sequence. The fourth dimension is associated with the third dimension, and the annotation information is used to represent the next indicator data that is immediately adjacent to the third indicator data sequence.
[0084] When dividing the training, validation, and test datasets, the preset ratios can be adjusted according to needs. For example, 70% of the data in the sample dataset can be used for training, 25% for testing, and 5% for validation.
[0085] As an optional implementation, when iteratively training the initial prediction model, the model training module can: for each training sample in the training dataset, use a preset number of convolutional long short-term memory networks to extract features from a preset number of fourth indicator data sequences in the training sample data, obtaining a preset number of associated features; stack the preset number of associated features with the third indicator data sequence in the training sample data along the channel dimension to obtain an input vector; input the input vector into a encoder / decoder to obtain a second prediction result output by the encoder / decoder, wherein the second prediction result is the predicted next indicator data immediately adjacent to the third indicator data sequence; and then process the annotation information in the training sample data... The first and second prediction results are input into the spatial feature discriminator to obtain the first and second classification results output by the spatial feature discriminator. The annotation information in the training sample data is combined with the third indicator data sequence to form the fifth indicator data sequence, and the second prediction result is combined with the third indicator data sequence to form the sixth indicator data sequence. The fifth and sixth indicator data sequences are input into the temporal feature discriminator to obtain the third and fourth classification results output by the temporal feature discriminator. A target loss function is constructed based on the first, second, third, and fourth classification results. During iterative training, the model parameters of the initial prediction model are adjusted according to the target loss function.
[0086] Optionally, when constructing the target loss function based on the first classification result, the second classification result, the third classification result, and the fourth classification result, a spatial distribution loss function can be constructed based on the first classification result and the second classification result; a sequence loss function can be constructed based on the third classification result and the fourth classification result; and the spatial distribution loss function and the sequence loss function can be weighted and summed to obtain the target loss function.
[0087] Before training begins, the model training module can set relevant training parameters such as the number of training rounds, the number of samples input to the model per run, and the learning rate, and then perform iterative training. During training, the discriminator parameters can be fixed first, allowing the generator's output to deceive the discriminator's standard. Then, the generator parameters can be fixed again, training the discriminator to successfully distinguish between output data and real data.
[0088] In the phase where the discriminator parameters are fixed, if the generator's output is distinguished from the real data by the spatial feature discriminator, it indicates that the generator's modeling of the structural distribution of the data to be predicted still has defects. Conversely, if the sequence containing the output is identified by the temporal feature discriminator, it proves that the generator's capture of the continuity features of the predicted sequence is still inaccurate. In the phase where the generator parameters are fixed, if the discriminator cannot distinguish between real data and generated data in terms of spatial distribution characteristics and temporal continuity, it indicates that the discriminator's judgment ability needs improvement. Through repeated training iterations, when the model generator and discriminator reach a Nash equilibrium, it indicates that the model has implicitly modeled the spatial structure and temporal features of the monitored data, and the model training is complete.
[0089] Finally, to verify the accuracy of the trained indicator data prediction model, the model training module can test the indicator data prediction model using the test dataset in the following manner: For each test sample data in the test dataset, the third and fourth indicator data sequences in the test sample data are input into the indicator data prediction model to obtain the third prediction result output by the indicator data prediction model; based on the third prediction result and the annotation information in the test sample data, test evaluation data is determined, wherein the type of test evaluation data includes at least one of the following: mean squared error (MSE), structural similarity (SSIM), and peak signal-to-noise ratio (PSNR); when the test evaluation data is greater than a preset threshold, the indicator data prediction model is deemed to have passed the test.
[0090] Throughout the training process, the memory and forgetting units of the Conv-LSTM network were used to capture the temporal features of sequences associated with the data to be predicted, effectively reducing the uncertainty of multi-dimensional contextual information in cloud service fault warning scenarios. At the same time, the indicator prediction loss function introduced adversarial learning and temporal consistency training strategies, comparing the model prediction results with the information structure distribution and temporal continuity of the real data, iteratively updating the model parameters, and exploring the intrinsic relationship between the temporal change patterns and spatial distribution characteristics of complex indicator data. The prediction results of the trained model eventually converged to the measured indicator data distribution, solving the problems of difficult model training and poor generalization ability in this scenario.
[0091] It should be noted that each module in the cloud service failure early warning device in this application embodiment corresponds one-to-one with each implementation step of the cloud service failure early warning method in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.
[0092] Example 3
[0093] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the cloud service fault early warning method in Embodiment 1 by running the computer program.
[0094] Specifically, the device containing the non-volatile storage medium executes the following steps by running the computer program: acquiring a first indicator data sequence of the first dimension and a second indicator data sequence of a preset number of groups of the second dimension from the cloud service, wherein the second dimension is associated with the first dimension; using a pre-trained indicator data prediction model to perform information distribution structure analysis and sequence continuity analysis on the first indicator data sequence and the second indicator data sequence to obtain a first prediction result, wherein the first prediction result is the predicted next indicator data that is immediately adjacent to the first indicator data sequence; when the first prediction result does not meet the preset indicator data threshold range, it is determined that there is an anomaly in the cloud service and an anomaly alarm message is issued.
[0095] According to an embodiment of this application, a processor is also provided for running a computer program, wherein the computer program executes the cloud service fault early warning method in embodiment 1 during runtime.
[0096] Specifically, the computer program executes the following steps during runtime: acquiring a first indicator data sequence of the first dimension and a second indicator data sequence of a preset number of groups of the second dimension from the cloud service, wherein the second dimension is associated with the first dimension; using a pre-trained indicator data prediction model to perform information distribution structure analysis and sequence continuity analysis on the first indicator data sequence and the second indicator data sequence to obtain a first prediction result, wherein the first prediction result is the predicted next indicator data that is immediately adjacent to the first indicator data sequence; when the first prediction result does not meet the preset indicator data threshold range, it is determined that there is an anomaly in the cloud service and an anomaly alarm message is issued.
[0097] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the cloud service fault early warning method of Embodiment 1 through the computer program.
[0098] Specifically, the processor is configured to execute the following steps via a computer program: acquire a first indicator data sequence of a first dimension and a second indicator data sequence of a preset number of second dimensions from the cloud service, wherein the second dimension is associated with the first dimension; perform information distribution structure analysis and sequence continuity analysis on the first indicator data sequence and the second indicator data sequence using a pre-trained indicator data prediction model to obtain a first prediction result, wherein the first prediction result is the predicted next indicator data that is immediately adjacent to the first indicator data sequence; when the first prediction result does not meet the preset indicator data threshold range, determine that there is an anomaly in the cloud service and issue an anomaly alarm message.
[0099] The sequence numbers of the above embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0100] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0101] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.
[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0105] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for early warning of cloud service failures, characterized in that, include: Obtain the first indicator data sequence of the first dimension and the second indicator data sequence of the second dimension of the preset number of groups in the cloud service, wherein the second dimension is associated with the first dimension; Using a pre-trained indicator data prediction model, information distribution structure analysis and sequence continuity analysis are performed on the first indicator data sequence and the second indicator data sequence to obtain a first prediction result, wherein the first prediction result is the predicted next indicator data that is immediately adjacent to the first indicator data sequence. When the first prediction result does not meet the preset indicator data threshold range, it is determined that the cloud service is abnormal and an abnormal alarm message is issued. The training process of the indicator data prediction model includes: An initial prediction model is constructed, wherein the initial prediction model is a generative adversarial network model, the generative adversarial network model includes a generator and a discriminator, the generator includes a preset number of convolutional long short-term memory networks and a codec, and the discriminator includes a spatial feature discriminator for information distribution structure analysis and a temporal feature discriminator for sequence continuity analysis. Obtain a sample dataset and divide the sample dataset into a training dataset, a validation dataset, and a test dataset according to a preset ratio. The sample dataset includes multiple sets of sample data obtained from the historical indicator data of the cloud service. Each set of sample data includes a third indicator data sequence of the third dimension, a fourth indicator data sequence of the fourth dimension of the preset number of sets, and annotation information corresponding to the third indicator data sequence. The fourth dimension is associated with the third dimension, and the annotation information is used to characterize the next indicator data that is immediately adjacent to the third indicator data sequence. The initial prediction model is iteratively trained using the training dataset, including: for each training sample data in the training dataset, using the preset number of convolutional long short-term memory networks to extract features from the preset number of fourth indicator data sequences in the training sample data, obtaining the preset number of associated features; stacking the preset number of associated features with the third indicator data sequence in the training sample data along the channel dimension to obtain an input vector; inputting the input vector into the encoder / decoder to obtain a second prediction result output by the encoder / decoder, wherein the second prediction result is the predicted next indicator data immediately adjacent to the third indicator data sequence in the training sample data; inputting the annotation information in the training sample data and the second prediction result into the spatial feature discriminator to obtain the... The spatial feature discriminator outputs a first classification result and a second classification result; a spatial distribution loss function is constructed based on the first classification result and the second classification result; the annotation information in the training sample data is combined with the third indicator data sequence to form a fifth indicator data sequence, and the second prediction result is combined with the third indicator data sequence to form a sixth indicator data sequence; the fifth indicator data sequence and the sixth indicator data sequence are respectively input into the temporal feature discriminator to obtain the third classification result and the fourth classification result output by the temporal feature discriminator; a sequence loss function is constructed based on the third classification result and the fourth classification result; the spatial distribution loss function and the sequence loss function are weighted and summed to obtain the target loss function; during iterative training, the model parameters of the initial prediction model are adjusted according to the target loss function; The hyperparameters of the initial prediction model are adjusted using the validation dataset to obtain the indicator data prediction model; The indicator data prediction model was tested using the test dataset.
2. The method according to claim 1, characterized in that, Constructing an initial prediction model includes: Construct a generator, wherein the codec in the generator includes: multiple downsampling convolutional layers, multiple depth residual blocks, and multiple upsampling convolutional layers; Construct a discriminator, wherein each discriminator includes: a plurality of downsampling modules, each downsampling module including: a downsampling convolutional layer and a linear unit with leakage correction; The initial prediction model is constructed based on the generator and the discriminator, and the generator parameters and discriminator parameters are determined based on the size of the data of the index to be predicted. The generator parameters include: the size of the downsampling layer convolution kernel, the size of the upsampling layer convolution kernel, and the size of the residual block. The discriminator parameters include: the size of the downsampling layer convolution kernel and the linear unit parameters with leakage correction.
3. The method according to claim 1, characterized in that, The indicator data prediction model is tested using the test dataset, including: For each test sample data in the test dataset, the third indicator data sequence and the fourth indicator data sequence in the test sample data are input into the indicator data prediction model to obtain the third prediction result output by the indicator data prediction model. Test evaluation data is determined based on the third prediction result and the annotation information in the test sample data, wherein the type of test evaluation data includes at least one of the following: mean squared error, structural similarity, peak signal-to-noise ratio; When the test evaluation data is greater than a preset threshold, the indicator data prediction model is deemed to have passed the test.
4. A cloud service fault early warning device, characterized in that, include: The acquisition module is used to acquire a first indicator data sequence of a first dimension and a second indicator data sequence of a preset number of second dimensions in the cloud service, wherein the second dimension is associated with the first dimension; The analysis module is used to perform information distribution structure analysis and sequence continuity analysis on the first indicator data sequence and the second indicator data sequence using a pre-trained indicator data prediction model to obtain a first prediction result, wherein the first prediction result is the predicted next indicator data that is immediately adjacent to the first indicator data sequence. The alarm module is used to determine that the cloud service is abnormal and issue an abnormal alarm message when the first prediction result does not meet the preset indicator data threshold range. The model training module is used to train the indicator data prediction model through the following process: An initial prediction model is constructed, wherein the initial prediction model is a generative adversarial network model, the generative adversarial network model includes a generator and a discriminator, the generator includes a preset number of convolutional long short-term memory networks and a codec, and the discriminator includes a spatial feature discriminator for information distribution structure analysis and a temporal feature discriminator for sequence continuity analysis. Obtain a sample dataset and divide the sample dataset into a training dataset, a validation dataset, and a test dataset according to a preset ratio. The sample dataset includes multiple sets of sample data obtained from the historical indicator data of the cloud service. Each set of sample data includes a third indicator data sequence of the third dimension, a fourth indicator data sequence of the fourth dimension of the preset number of sets, and annotation information corresponding to the third indicator data sequence. The fourth dimension is associated with the third dimension, and the annotation information is used to characterize the next indicator data that is immediately adjacent to the third indicator data sequence. The initial prediction model is iteratively trained using the training dataset, including: for each training sample data in the training dataset, using the preset number of convolutional long short-term memory networks to extract features from the preset number of fourth indicator data sequences in the training sample data, obtaining the preset number of associated features; stacking the preset number of associated features with the third indicator data sequence in the training sample data along the channel dimension to obtain an input vector; inputting the input vector into the encoder / decoder to obtain a second prediction result output by the encoder / decoder, wherein the second prediction result is the predicted next indicator data immediately adjacent to the third indicator data sequence in the training sample data; inputting the annotation information in the training sample data and the second prediction result into the spatial feature discriminator to obtain the... The spatial feature discriminator outputs a first classification result and a second classification result; a spatial distribution loss function is constructed based on the first classification result and the second classification result; the annotation information in the training sample data is combined with the third indicator data sequence to form a fifth indicator data sequence, and the second prediction result is combined with the third indicator data sequence to form a sixth indicator data sequence; the fifth indicator data sequence and the sixth indicator data sequence are respectively input into the temporal feature discriminator to obtain the third classification result and the fourth classification result output by the temporal feature discriminator; a sequence loss function is constructed based on the third classification result and the fourth classification result; the spatial distribution loss function and the sequence loss function are weighted and summed to obtain the target loss function; during iterative training, the model parameters of the initial prediction model are adjusted according to the target loss function; The hyperparameters of the initial prediction model are adjusted using the validation dataset to obtain the indicator data prediction model; The indicator data prediction model was tested using the test dataset.
5. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the cloud service fault early warning method according to any one of claims 1 to 3 by running the computer program.
6. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the cloud service fault early warning method according to any one of claims 1 to 3 through the computer program.