Operation and maintenance data prediction method, device, equipment and product

Abstract

The invention discloses an operation and maintenance data prediction method, device, equipment and product, and the method comprises the steps: training a long short-term memory (LSTM) neural network model by employing corrected historical operation and maintenance time sequence data; the method has the advantages that future operation and maintenance time series data are predicted by the aid of the trained LSTM neural network model, the method is high in operation and maintenance data prediction speed, accurate in prediction effect and capable of effectively improving operation and maintenance quality, an operation and maintenance team can actively prevent faults according to the predicted operation and maintenance data, equipment downtime is shortened, and operation and maintenance efficiency is improved. And stable operation of services is ensured.

Description

Technical Field

[0001] The present invention belongs to the field of operation and maintenance management, and in particular relates to a method, device, equipment and product for predicting operation and maintenance data. Background Art

[0002] In the daily operation and maintenance of digitally intelligent enterprises, O&M target monitoring can include hardware monitoring, system monitoring, application monitoring, and network performance monitoring. Most monitoring data is collected in the form of time series. Under existing technical conditions, O&M personnel typically monitor the current status of important time series indicators and, based on historical data trends, manually predict the future state of the performance indicators of the target monitored objects. This results in high labor consumption, difficulty ensuring long-term continuous judgment, slow prediction speed, and low judgment accuracy. Summary of the Invention

[0003] In view of this, the present invention aims to overcome the defects in the prior art and proposes a method, device, equipment and product for predicting operation and maintenance data.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0005] In a first aspect, the present invention discloses a method for predicting operation and maintenance data, comprising: obtaining historical operation and maintenance time series data of a target monitoring object, wherein the historical operation and maintenance time series data is historical time series data of performance indicators of the target monitoring object;

[0006] The historical operation and maintenance time series data is divided into periodic time series data and irregular time series data. The periodic time series data is the time series data that presents periodic changes, and the irregular time series data is the time series data that does not present periodic changes.

[0007] Perform wavelet transform on irregular time series data and recombine them to make them show periodic changes, thus obtaining the corrected historical operation and maintenance time series data.

[0008] The long short-term memory neural network model LSTM is trained using the corrected historical operation and maintenance time series data to obtain a trained long short-term memory neural network model LSTM;

[0009] Based on the input operation and maintenance time series data, the trained long short-term memory neural network model LSTM is used to predict future operation and maintenance time series data, where the operation and maintenance time series data is the time series data of the performance indicators of the target monitoring object.

[0010] In one embodiment of the present invention, historical operation and maintenance time series data is divided into periodic time series data and irregular time series data, including: performing Fourier transform on the historical operation and maintenance time series data to identify the periodic time series data and irregular time series data in the historical operation and maintenance time series data.

[0011] In one embodiment of the present invention, historical operation and maintenance time series data is divided into periodic time series data and irregular time series data, including: performing multi-granularity visualization processing on the historical operation and maintenance time series data, and dividing the time series data that presents repetitive waveforms into periodic time series data; and dividing the time series data that does not present repetitive waveforms into irregular time series data.

[0012] In one embodiment of the present invention, before dividing the historical operation and maintenance time series data into periodic time series data and irregular time series data, the method further includes: performing smoothing and denoising processing on the historical operation and maintenance time series data.

[0013] In one embodiment of the present invention, performing smoothing and denoising processing on the historical operation and maintenance time series data includes: performing smoothing and denoising processing on the historical operation and maintenance time series data using a least squares smoothing filter algorithm.

[0014] In one embodiment of the present invention, predicting future operation and maintenance time series data includes: displaying the prediction results in the form of charts or reports.

[0015] In one embodiment of the present invention, obtaining historical operation and maintenance time series data of a target monitoring object includes: storing the historical operation and maintenance time series data in a time series database.

[0016] In a second aspect, the present invention discloses a device for predicting operation and maintenance data, the device comprising:

[0017] A historical data acquisition module is used to acquire historical operation and maintenance time series data of a target monitoring object, wherein the historical operation and maintenance time series data is historical time series data of performance indicators of the target monitoring object;

[0018] A data partitioning module is used to divide historical operation and maintenance time series data into periodic time series data and irregular time series data, wherein periodic time series data is time series data that presents periodic changes, and irregular time series data is time series data that does not present periodic changes;

[0019] The data correction module is used to perform wavelet transformation and recombine irregular time series data to make the irregular time series data show periodic changes and obtain the corrected historical operation and maintenance time series data;

[0020] The model training module is used to train the long short-term memory neural network model LSTM using the corrected historical operation and maintenance time series data to obtain a trained long short-term memory neural network model LSTM;

[0021] The prediction module is used to predict future operation and maintenance time series data based on the input operation and maintenance time series data using the trained long short-term memory neural network model LSTM, where the operation and maintenance time series data is the time series data of the performance indicators of the target monitoring object.

[0022] In a third aspect, the present invention discloses an electronic device comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0023] In a fourth aspect, the present invention discloses a computer program product, comprising a computer program, which implements the above method when executed by a processor.

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] The present invention discloses a method, device, equipment and product for predicting operation and maintenance data, comprising: obtaining historical operation and maintenance time series data of a target monitoring object; dividing the historical operation and maintenance time series data into periodic time series data and irregular time series data; performing wavelet transform on the irregular time series data and recombining the irregular time series data so that the irregular time series data presents periodic changes, thereby obtaining corrected historical operation and maintenance time series data; using the corrected historical operation and maintenance time series data to train a long short-term memory neural network model LSTM; based on the input operation and maintenance time series data, using the trained long short-term memory neural network model LSTM to predict future operation and maintenance time series data. The present invention discloses a method, device, equipment and product for predicting operation and maintenance data, using the corrected historical operation and maintenance time series data to train a long short-term memory neural network model LSTM; using the trained long short-term memory neural network model LSTM to predict future operation and maintenance time series data, which has the characteristics of fast operation and maintenance data prediction speed, accurate prediction effect and the ability to effectively improve operation and maintenance quality. The operation and maintenance team can proactively prevent equipment failures, reduce equipment downtime and ensure stable business operation based on the predicted operation and maintenance data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] In the attached figure:

[0028] Figure 1 A schematic diagram of a method for predicting operation and maintenance data according to an embodiment of the present invention;

[0029] Figure 2This is a data partitioning diagram of a method for predicting operation and maintenance data according to an embodiment of the present invention;

[0030] Figure 3 A schematic diagram of smoothing and denoising of a prediction method for operation and maintenance data according to an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of a device for predicting operation and maintenance data according to an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of an electronic device for predicting operation and maintenance data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0034] In the description of the present invention, it should be further clarified that the terms "first," "second," etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first," "second," etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0035] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0036] In one embodiment of the present invention, Figure 1 As shown, a method for predicting operation and maintenance data includes:

[0037] Step S101: Acquire historical operation and maintenance time series data of a target monitoring object, wherein the historical operation and maintenance time series data is historical time series data of performance indicators of the target monitoring object;

[0038] For example, the monitoring system obtains historical operation and maintenance time series data generated by the target monitoring object (such as a server, network equipment, etc.), such as any one of CPU usage, memory usage, disk I / O, and network traffic. Data collection can be achieved through existing monitoring tools (such as Prometheus, Zabbix), etc. The collected data is stored in the form of a time series.

[0039] For example, obtaining the historical operation and maintenance time series data of the target monitoring object is to obtain the CPU usage time series data of the past year from the server cluster of a data center, with an interval of 1 minute.

[0040] In this embodiment, the historical operation and maintenance time series data is stored in a time series database.

[0041] For example, the acquired historical operation and maintenance time series data is transmitted through the Kafka message queue. As a high-throughput distributed messaging system, Kafka can ensure the real-time and reliability of the data. Historical operation and maintenance time series data is stored in the ClickHouse time series database through the Kafka channel. The ClickHouse time series database is a high-performance columnar database that is particularly suitable for storing and querying large-scale time series data. During the storage process, the data is partitioned by timestamp and performance indicator type to improve query efficiency.

[0042] In this embodiment, the time series database may also be a MySQL database.

[0043] In this embodiment, if Figure 3 As shown in the figure, the historical operation and maintenance time series data is smoothed and denoised to remove noise in the data.

[0044] Exemplarily, the least squares smoothing filter algorithm is used to smooth and denoise the historical operation and maintenance time series data.

[0045] The smoothing and denoising process is as follows:

[0046] The historical operation and maintenance time series data is segmented, and the length of each segment is N.

[0047] Fit a least squares curve to each segment of data to obtain smoothed data points.

[0048] The smoothed data points are used to replace the noise points in the original historical operation and maintenance time series data to obtain the denoised historical operation and maintenance time series data.

[0049] Step S102: Figure 1 and Figure 2 As shown, historical operation and maintenance time series data is divided into periodic time series data and irregular time series data, wherein periodic time series data is time series data showing periodic changes, and irregular time series data is time series data not showing periodic changes;

[0050] For example, periodic time series data refers to data that fluctuates repeatedly within a certain time range, and its changes are regular and predictable. Therefore, the long short-term memory neural network model LSTM is trained using periodic time series data to achieve better predictability of the model.

[0051] Step S103: Perform wavelet transform on the irregular time series data and recombine them so that the irregular time series data presents periodic changes, thereby obtaining corrected historical operation and maintenance time series data;

[0052] In this embodiment, the process of performing wavelet transformation and recombining irregular time series data is as follows:

[0053] For irregular time series data (such as sudden abnormal data), wavelet decomposition is performed to decompose the irregular time series data into approximate coefficients and detail coefficients, and then split into components of different frequencies, each component corresponding to a specific signal feature;

[0054] The components are tested for stationarity using ADF or KPSS. If the components are stationary, the stationary component data are recombined to replace the original irregular time series data.

[0055] If the component still does not have stationarity, the wavelet decomposition is repeated until the decomposed component data has stationarity, and then the stationary component data is recombined to replace the original irregular time series data.

[0056] In this embodiment, the wavelet transform recombination method is used to make the irregular time series data show periodic changes, and then the long short-term memory neural network model LSTM is trained using the corrected historical operation and maintenance time series data, which further improves the prediction accuracy of the model, provides a scientific basis for the operation and maintenance management of servers and other equipment, and helps to improve the operating efficiency and stability of the equipment.

[0057] Step S104: Using the corrected historical operation and maintenance time series data, the long short-term memory neural network model LSTM is trained to obtain a trained long short-term memory neural network model LSTM;

[0058] In this embodiment, the training process of the long short-term memory neural network model LSTM is as follows:

[0059] The corrected historical operation and maintenance time series data is divided into a training set and a validation set, with the training set accounting for 80% and the validation set accounting for 20%;

[0060] Determine the initial parameters of the long short-term memory neural network model LSTM, including: the number of hidden layer units, learning rate, number of training times, etc.;

[0061] The LSTM neural network model is trained using the training set. During the training process, the model parameters are adjusted using the backpropagation algorithm. The learning rate and number of training times are adjusted according to the loss function value of the validation set until the LSTM neural network model is trained.

[0062] Use the validation set to evaluate the trained long short-term memory neural network model LSTM and calculate the prediction error (such as mean square error, mean absolute error, etc.) to ensure the prediction accuracy of the model.

[0063] Step S105: Based on the input operation and maintenance time series data, the trained long short-term memory neural network model LSTM is used to predict future operation and maintenance time series data, where the operation and maintenance time series data is the time series data of the performance indicators of the target monitoring object.

[0064] In this embodiment, the input operation and maintenance time series data is pre-processed into the same format as the historical operation and maintenance time series data, including timestamps, the same target monitoring object performance indicators, etc.

[0065] For example, the target monitoring object performance indicator is CPU usage, and the input operation and maintenance time series data is also CPU usage time series data.

[0066] In this embodiment, the prediction results are presented in the form of charts or reports.

[0067] Based on the previous embodiment, in another implementation of the present invention, the historical operation and maintenance time series data is divided into periodic time series data and irregular time series data, including: performing Fourier transform on the historical operation and maintenance time series data, and identifying the periodic time series data and irregular time series data in the historical operation and maintenance time series data.

[0068] In another embodiment of the present invention, historical operation and maintenance time series data is divided into periodic time series data and irregular time series data, including: performing multi-granularity visualization processing on the historical operation and maintenance time series data, and dividing the time series data that presents repetitive waveforms into periodic time series data; and dividing the time series data that does not present repetitive waveforms into irregular time series data.

[0069] In this embodiment, multi-granularity visualization processing is performed on historical operation and maintenance time series data, including: using short-term view, medium-term view and long-term view to visualize the corresponding historical operation and maintenance time series data, and further determining the historical operation and maintenance time series data that presents repetitive waveforms.

[0070] like Figure 4 As shown, the present invention also discloses a prediction device for operation and maintenance data, comprising:

[0071] The historical data acquisition module 401 is used to acquire historical operation and maintenance time series data of the target monitoring object, wherein the historical operation and maintenance time series data is the historical time series data of the performance indicators of the target monitoring object;

[0072] The data partitioning module 402 is used to partition the historical operation and maintenance time series data into periodic time series data and irregular time series data, wherein the periodic time series data is time series data that presents periodic changes, and the irregular time series data is time series data that does not present periodic changes;

[0073] The data correction module 403 is used to perform wavelet transformation and recombine the irregular time series data so that the irregular time series data presents periodic changes and obtains corrected historical operation and maintenance time series data;

[0074] The model training module 404 is used to train the long short-term memory neural network model LSTM using the corrected historical operation and maintenance time series data to obtain a trained long short-term memory neural network model LSTM;

[0075] The prediction module 405 is used to predict future operation and maintenance time series data based on the input operation and maintenance time series data using the trained long short-term memory neural network model LSTM, wherein the operation and maintenance time series data is the time series data of the performance indicators of the target monitoring object.

[0076] The present invention also discloses an electronic device, such as Figure 5 As shown, an embodiment is disclosed, which is a block diagram of an electronic device suitable for the above-mentioned operation and maintenance data prediction device.

[0077] The electronic device 50 of this embodiment includes a processor 501, which can perform various appropriate actions and processes according to the program stored in the ROM 502 or the program loaded from the storage part 508 into the RAM 503. The processor 501 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a dedicated microprocessor, etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present invention.

[0078] RAM 503 stores various programs and data required for the operation of electronic device 50. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes the programs in ROM 502 and / or RAM 503 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than ROM 502 and RAM 503, and processor 501 may also execute the programs stored in one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0079] According to an embodiment of the present invention, the electronic device 50 may further include an I / O interface 505, which is also connected to the bus 504. The electronic device 50 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube, a liquid crystal display, and a speaker; a storage portion 508 including a hard disk; and a communication portion 509 including a network interface card such as a LAN card or a modem. The communication portion 509 performs communication processing via a network such as the Internet. A drive 5010 is also connected to the I / O interface 505 as needed. A removable medium 5011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in the drive 5010 as needed, so that a computer program read therefrom can be installed into the storage portion 508 as needed.

[0080] The present invention also provides a computer-readable storage medium.

[0081] The computer-readable storage medium may be included in the electronic device / device system described in the above embodiments, or may exist independently and not be incorporated into the electronic device / device. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of the present invention.

[0082] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0083] Embodiments of the present invention also include a computer program product.

[0084] The computer program product includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiment of the present invention.

[0085] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal over a network medium. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0086] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written by any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages. Programming languages ​​include, but are not limited to, Java, C++, Python, C language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network or a wide area network, or can be connected to an external computing device.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments and / or claims of the present invention may be combined and / or coupled in various ways, and all such combinations and / or couplings fall within the scope of the present invention.

[0088] The embodiments of the present invention have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described above separately, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The scope of the present invention is defined by the appended claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for predicting operation and maintenance data, characterized in that: include: Acquire historical operation and maintenance time series data of the target monitoring object, wherein the historical operation and maintenance time series data is historical time series data of performance indicators of the target monitoring object; Dividing the historical operation and maintenance time series data into periodic time series data and irregular time series data, wherein the periodic time series data is time series data showing periodic changes, and the irregular time series data is time series data not showing periodic changes; Performing wavelet transformation and recombining the irregular time series data so that the irregular time series data presents periodic changes, thereby obtaining the corrected historical operation and maintenance time series data; Using the corrected historical operation and maintenance time series data, training a long short-term memory neural network model LSTM to obtain the trained long short-term memory neural network model LSTM; Based on the input operation and maintenance time series data, the trained long short-term memory neural network model LSTM is used to predict the future operation and maintenance time series data, wherein the operation and maintenance time series data is the time series data of the performance indicators of the target monitoring object.

2. The method for predicting operation and maintenance data according to claim 1, characterized in that: The dividing the historical operation and maintenance time series data into periodic time series data and irregular time series data includes: performing Fourier transform on the historical operation and maintenance time series data to identify the periodic time series data and the irregular time series data in the historical operation and maintenance time series data.

3. The method for predicting operation and maintenance data according to claim 1, characterized in that: The dividing of the historical operation and maintenance time series data into periodic time series data and irregular time series data includes: performing multi-granularity visualization processing on the historical operation and maintenance time series data, dividing the time series data that presents repetitive waveforms into the periodic time series data; and dividing the time series data that does not present repetitive waveforms into the irregular time series data.

4. The method for predicting operation and maintenance data according to claim 1, characterized in that: Before dividing the historical operation and maintenance time series data into periodic time series data and irregular time series data, the method further includes: performing smoothing and denoising processing on the historical operation and maintenance time series data.

5. The method for predicting operation and maintenance data according to claim 4, characterized in that: The smoothing and denoising process is performed on the historical operation and maintenance time series data, comprising: performing smoothing and denoising process on the historical operation and maintenance time series data using a least squares smoothing filter algorithm.

6. The method for predicting operation and maintenance data according to claim 1, characterized in that: The predicting of the future operation and maintenance time series data includes: displaying the prediction results in the form of charts or reports.

7. The method for predicting operation and maintenance data according to claim 1, characterized in that: The obtaining of historical operation and maintenance time series data of the target monitoring object includes: storing the historical operation and maintenance time series data in a time series database.

8. A device for predicting operation and maintenance data, characterized by: The device comprises: A historical data acquisition module is used to acquire historical operation and maintenance time series data of a target monitoring object, wherein the historical operation and maintenance time series data is historical time series data of performance indicators of the target monitoring object; a data partitioning module, configured to partition the historical operation and maintenance time series data into periodic time series data and irregular time series data, wherein the periodic time series data is time series data that exhibits periodic changes, and the irregular time series data is time series data that does not exhibit periodic changes; A data correction module is used to perform wavelet transformation and recombine the irregular time series data so that the irregular time series data presents periodic changes, thereby obtaining the corrected historical operation and maintenance time series data; A model training module is used to train a long short-term memory neural network model LSTM using the corrected historical operation and maintenance time series data to obtain the trained long short-term memory neural network model LSTM; The prediction module is used to predict the future operation and maintenance time series data based on the input operation and maintenance time series data using the trained long short-term memory neural network model LSTM, wherein the operation and maintenance time series data is the time series data of the performance indicators of the target monitoring object.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images