Operation and maintenance index monitoring model training method and device and data filtering method

Abstract

The application discloses an operation and maintenance index monitoring model training method and device and a data filtering method. The method comprises the following steps: determining a first training data set; determining a target filter, wherein the target filter comprises a space kernel, a value range kernel and a time domain kernel; filtering the first training data set based on the target filter to obtain a second training data set; and training an operation and maintenance index monitoring model based on the second training data set. The application solves the technical problem of poor model training effect caused by the fact that the prior art cannot effectively reduce the adverse effect of fluctuation noise data on the model.

Description

Technical Field

[0001] The present invention relates to the field of IT operation and maintenance, and in particular to a method and device for training an operation and maintenance indicator monitoring model and a data filtering method. Background Art

[0002] Currently, traditional enterprise O&M tools, such as Zabbix, Prometheus, and SolarWinds, are generally based on threshold-based alerts. This means that when the alarm occupancy rate exceeds a certain threshold and is triggered for a period of time, the O&M personnel are notified via SMS, phone calls, or instant messaging interfaces. This type of alerting method suffers from low alert effectiveness, high levels of noise, and a high number of duplicate alerts. For example, in the following scenario:

[0003] Some indicators fluctuate repeatedly around the monitoring threshold in the short term, which can easily generate a large number of repeated alarms.

[0004] Some monitoring indicators degrade very slowly. For example, it takes more than a month or even longer for memory usage to increase from 90% to 95%. Setting the monitoring threshold to 90% will continue to generate a large number of repeated alarms.

[0005] The above low-value or even invalid alarms are collectively referred to as noise alarms in this patent. In short, a large number of repeated and low-value alarms will seriously reduce the sensitivity of operation and maintenance personnel, and may cause them to miss key alarm information.

[0006] Although there are some methods in the prior art that can be used to suppress alarm noise, they have the following shortcomings:

[0007] 1) The prediction model predetermines the function model and individual parameter values ​​of the monitoring indicators. It is often the case that the prediction results do not match the actual distribution of the monitoring indicators, and the effect of noise suppression is poor.

[0008] 2) It is difficult to make effective predictions for monitoring indicators in time windows of different sizes, and usually only monitoring indicators within a certain time limit can be predicted. The above patent authorization is based on predictions in a single time window, which makes it difficult to simultaneously take into account the prediction capabilities of short-, medium-, and long-term faults. For example, some patented methods have good prediction effects on hourly data sets and can predict the changing trends of monitoring indicators in the next few hours. However, it is difficult to make effective predictions for monitoring indicators that deteriorate severely in a short period of time or slowly deteriorate within a few days, or the prediction accuracy is significantly different from the actual situation.

[0009] In summary, traditional techniques such as time series analysis and regression analysis are limited by model capabilities and have shortcomings in predictive capabilities. On the other hand, while recurrent neural network (LSTM / GRU) models excel in areas such as time series data prediction and natural language processing (NLP), they face the following challenges in IT monitoring and prediction scenarios:

[0010] Frequent fluctuations in some monitoring indicators can easily affect the gating and model state of recurrent neural networks, affecting prediction accuracy. For example, when a server starts and shuts down processes, indicators such as CPU and memory will fluctuate frequently, interfering with traditional recurrent network models.

[0011] In a large time window, as the training data increases, the number of parameters that need to be trained also increases, and the time consumption of model training increases, which is not conducive to rapid testing, prediction and response in the production environment.

[0012] Secondly, a common problem faced by recurrent time networks is that if the sample points are collected at different intervals, the prediction effect will be very poor. In actual production, if the data collection intervals are uneven due to reasons such as failures in the collection system, the prediction accuracy will be seriously reduced.

[0013] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0014] The embodiments of the present invention provide an operation and maintenance indicator monitoring model training method, device and data filtering method to at least solve the technical problem of poor model training effect caused by the inability of the existing technology to effectively reduce the adverse effects of fluctuating noise data on the model.

[0015] According to one aspect of an embodiment of the present invention, a method for training an operation and maintenance indicator monitoring model is provided, comprising: determining a first training data set; determining a target filter, wherein the target filter includes a spatial kernel, a range kernel, and a time domain kernel; filtering the first training data set based on the target filter to obtain a second training data set; and training the operation and maintenance indicator monitoring model based on the second training data set.

[0016] Optionally, determining the first training data set includes: determining the size of the prediction window of the operation and maintenance indicator monitoring model; determining multiple samples, and a sampling period corresponding to each sample in the multiple samples, wherein the sample is a server; determining the sampling point corresponding to each sample in the multiple samples, wherein the number of sampling points corresponding to each sample is the same as the size of the prediction window, and the sampling point includes a timestamp representing the sampling time point, and an index value of the target operation and maintenance index of the sample sampled at the sampling time point, wherein the target operation and maintenance index is an index monitored by the operation and maintenance indicator monitoring model; determining a label corresponding to each sample in the multiple samples; determining a sample set and a label set in the first training data set, wherein the sample set includes a sampling point corresponding to each sample in the multiple samples, and the label set includes a label corresponding to each sample in the multiple samples.

[0017] Optionally, determining the target filter includes: determining a spatial objective function, a range objective function, and a time domain objective function; constructing a weight specification term of the target filter based on the spatial objective function, the range objective function, and the time domain objective function, wherein the spatial part of the spatial objective function and the weight specification term components is a spatial kernel, the range part of the range objective function and the weight specification term components is a range kernel, and the time domain part of the time domain objective function and the weight specification term components is a time domain kernel; constructing a target filter based on the weight specification term, the spatial objective function, the range objective function, and the time domain objective function, wherein the target filter is used to perform bilateral filtering on the first training data set.

[0018] Optionally, the target filter is used to reduce noise on a first training data set, wherein the spatial kernel is used to reduce noise on a first sample in the first training data set, wherein the first sample is a sample in the first training data set whose sample value variance is less than a first preset variance threshold; the range kernel is used to reduce noise on a second sample in the first training data set, wherein the second sample is a sample in the first training data set whose sample value variance is greater than a first preset variance threshold; and the time domain kernel is used to reduce noise on a third sample in the first training data set, wherein the third sample is a sample in the first training data set whose sampling period variance is greater than a second preset variance threshold.

[0019] Optionally, filtering the training data set based on the target filter to obtain the second training data set includes: determining a filtering step size of the target filter, wherein the filtering step size is greater than 1; and downsampling the training data set based on the target filter according to the filtering step size to obtain the second training data set.

[0020] Optionally, filtering the first training data set based on the target filter to obtain the second training data set includes: determining a sample set matrix corresponding to the sample set in the first training data set, and a label set matrix corresponding to the label set of the samples in the first training data set, wherein the elements in the sample set matrix are sampling points corresponding to each sample, the sampling point includes a timestamp representing the sampling time point, and an indicator value of the target operation and maintenance indicator of the sample sampled at the sampling time point, the target operation and maintenance indicator is an indicator monitored by the operation and maintenance indicator monitoring model, and the elements in the label set matrix are labels corresponding to each sample; merging the sample set matrix and the label set matrix into a first augmented matrix corresponding to the first training data set; filtering the first augmented matrix based on the target filter to obtain a second augmented matrix, and using the second augmented matrix as the second training data set.

[0021] According to another aspect of an embodiment of the present invention, a data filtering method is also provided, including: determining a training data set; determining a target filter, wherein the target filter includes a spatial kernel, a value range kernel and a time domain kernel; when the sample in the training data set is a first sample, using the spatial kernel to reduce the noise of the first sample in the training data set, wherein the first sample is a sample in the training data set whose sample value variance is less than a first preset variance threshold; when the sample in the training data set is a second sample, using the value range kernel to reduce the noise of the second sample in the training data set, wherein the second sample is a sample in the training data set whose sample value variance is greater than the first preset variance threshold; when the sample in the training data set is a third sample, using the time domain kernel to reduce the noise of the third sample in the training data set, wherein the third sample is a sample in the training data set whose sampling period variance is greater than the second preset variance threshold.

[0022] According to another aspect of an embodiment of the present invention, an operation and maintenance indicator monitoring model training device is also provided, including: a first processing module for determining a first training data set; a second processing module for determining a target filter, wherein the target filter includes a spatial kernel, a value range kernel and a time domain kernel; a filtering module for filtering the first training data set based on the target filter to obtain a second training data set; and a training module for training the operation and maintenance indicator monitoring model based on the second training data set.

[0023] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided. The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute an operation and maintenance indicator monitoring model training method.

[0024] According to another aspect of an embodiment of the present invention, an electronic device is provided. The electronic device includes a processor, and the processor is used to run a program, wherein the operation and maintenance indicator monitoring model training method is executed when the program is running.

[0025] In an embodiment of the present invention, a first training data set is determined; a target filter is determined, wherein the target filter includes a spatial kernel, a value range kernel and a time domain kernel; the first training data set is filtered based on the target filter to obtain a second training data set; and an operation and maintenance indicator monitoring model is trained based on the second training data set. By constructing a filter including a spatial kernel, a value range kernel and a time domain kernel to filter the training data set, the purpose of effectively reducing the fluctuating noise data in the training data set is achieved, thereby achieving the technical effect of improving the training effect of the operation and maintenance indicator monitoring model, and further solving the technical problem of poor model training effect caused by the inability to effectively reduce the adverse effects of fluctuating noise data on the model in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 1 is a flow chart of a method for training an operation and maintenance indicator monitoring model according to an embodiment of the present invention;

[0028] Figure 2 is a flow chart of a data filtering method provided according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of an operation and maintenance indicator monitoring model provided according to an embodiment of the present invention;

[0030] Figure 4 It is a structural diagram of a computer device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] According to an embodiment of the present invention, a method embodiment of an operation and maintenance indicator monitoring model training method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] Figure 1 This is an operation and maintenance indicator monitoring model training method according to an embodiment of the present invention, such as Figure 1 As shown, the method includes the following steps:

[0035] Step S102, determining a first training data set;

[0036] In some embodiments of the present application, determining the first training data set includes: determining the size of the prediction window of the operation and maintenance indicator monitoring model; determining multiple samples, and a sampling period corresponding to each sample of the multiple samples, wherein the sample is a server; determining the sampling points corresponding to each sample of the multiple samples, wherein the number of sampling points corresponding to each sample is the same as the size of the prediction window, and the sampling points include a timestamp representing the sampling time point, and an index value of the target operation and maintenance index of the sample obtained by sampling at the sampling time point, wherein the target operation and maintenance index is the index monitored by the operation and maintenance indicator monitoring model; determining the label corresponding to each sample of the multiple samples; determining the sample set and label set in the first training data set, wherein the sample set includes the sampling point corresponding to each sample of the multiple samples, and the label set includes the label corresponding to each sample of the multiple samples.

[0037] In some embodiments of the present application, the sample set and label set in the first training dataset are as follows:

[0038]

[0039] The sample set X contains n samples, each of which contains m sample points with a fixed time sampling period, that is, the prediction window size of the recurrent neural network is m, Y label is the label of n samples.

[0040] Step S104, determining a target filter, wherein the target filter includes a spatial kernel, a range kernel, and a time domain kernel;

[0041] In some embodiments of the present application, determining the target filter includes: determining a spatial objective function, a range objective function, and a time domain objective function; constructing a weight specification term of the target filter based on the spatial objective function, the range objective function, and the time domain objective function, wherein the spatial portion of the spatial objective function and the weight specification term components is the spatial kernel, the range portion of the range objective function and the weight specification term components is the range kernel, and the time domain portion of the time domain objective function and the weight specification term components is the time domain kernel; constructing the target filter based on the weight specification term, the spatial objective function, the range objective function, and the time domain objective function, wherein the target filter is used to perform bilateral filtering on the first training data set.

[0042] In some embodiments of the present application, when the prediction window m is large, the training and prediction efficiency of the recurrent neural network decreases significantly. At the same time, short-term drastic fluctuations in data are also not conducive to the prediction accuracy of the network. In this case, it is necessary to use the filter described in the following formula to filter and denoise the data:

[0043]

[0044]

[0045] In the above formula, σ s , σ I , σ t The kernels are spatial, value, and time domains, respectively. G is a Gaussian function, W is a weight normalization term, S is the set of filter elements within the filter, j and i are elements within S, and j is the corresponding timestamp of the sample. This filter is based on matrix distance, sample value difference, and sample time difference, and consists of a spatial kernel, a value domain kernel, and a time domain kernel.

[0046] It should be noted that, in some embodiments of the present application, G may also be other functions that can eliminate data noise.

[0047] Step S106, filtering the first training data set based on the target filter to obtain a second training data set;

[0048] In some embodiments of the present application, the target filter is used to denoise the first training data set, wherein the spatial kernel is used to denoise a first sample in the first training data set, the first sample being a sample in the first training data set with a sample value variance less than a first preset variance threshold; the value domain kernel is used to denoise a second sample in the first training data set, the second sample being a sample in the first training data set with a sample value variance greater than a first preset variance threshold; and the time domain kernel is used to denoise a third sample in the first training data set, the third sample being a sample in the first training data set with a sampling period variance greater than a second preset variance threshold.

[0049] Specifically, when the monitored sample value within the filter does not change significantly over time, plays a role, which is equivalent to simply Gaussian smoothing denoising of the sample. When the monitored sample value within the filter changes significantly, i.e., the monitored index fluctuates sharply, plays a role, which can maintain the fluctuation characteristics while smoothing the noise effect. When the sampling time difference of the sample within the filter is relatively obvious, i.e., the time difference of the monitored sample is large, plays a role, which can maintain the difference characteristics of the sample in the time dimension while smoothing the noise effect.

[0050] In some embodiments of the present application, filtering the first training data set based on the target filter to obtain a second training data set includes: determining a sample set matrix corresponding to a sample set in the first training data set and a label set matrix corresponding to a label set of the samples in the first training data set, wherein an element in the sample set matrix is a sampling point corresponding to each sample, the sampling point includes a timestamp representing a sampling time point and an index value of a target operation and maintenance index of the sample sampled at the sampling time point, the target operation and maintenance index being an index monitored by the operation and maintenance index monitoring model, and an element in the label set matrix being a label corresponding to each sample; merging the sample set matrix and the label set matrix into a first augmented matrix corresponding to the first training data set; filtering the first augmented matrix based on the target filter to obtain a second augmented matrix, and taking the second augmented matrix as the second training data set.

[0051] In some embodiments of the present application, the augmented matrix corresponding to the first training data set is as follows:

[0052]

[0053] In some embodiments of the present application, filtering the training data set based on the target filter to obtain a second training data set includes: determining a filtering step size of the target filter, wherein the filtering step size is greater than 1; and downsampling the training data set based on the target filter according to the filtering step size to obtain the second training data set.

[0054] Specifically, the above filter can also be used to perform filtering operation on the augmented matrix, and the filtering step size is set to k (k>1), which can satisfy the sampling operation and obtain the matrix Its form is:

[0055]

[0056] In the formula, m>m′n>n′, that is yes The downsampling result of . Using it as a sample set to predict IT operation and maintenance indicators can improve prediction accuracy and efficiency.

[0057] Step S108: training the operation and maintenance indicator monitoring model based on the second training data set.

[0058] In some embodiments of the present application, the above-mentioned operation and maintenance indicator monitoring model can be a recurrent neural network model.

[0059] In some embodiments of the present application, training the operation and maintenance indicator monitoring model based on the second training data set includes: determining a sample set matrix corresponding to the sample set in the second training data set, and a label set matrix corresponding to the label set of the samples in the second training data set, wherein the elements in the sample set matrix are sampling points corresponding to each sample, the sampling points include a timestamp representing the sampling time point, and an indicator value of the target operation and maintenance indicator of the sample obtained by sampling at the sampling time point, the target operation and maintenance indicator is the indicator monitored by the operation and maintenance indicator monitoring model, and the elements in the label set matrix are the labels corresponding to each sample; merging the sample set matrix and the label set matrix into an augmented matrix; and inputting the augmented matrix into the operation and maintenance indicator monitoring model for training.

[0060] To further elaborate Figure 1 The operation and maintenance indicator monitoring model training method shown in , takes the operation and maintenance indicator as the server memory usage as an example, Figure 1 The method shown in is further explained.

[0061] Specifically, let's take the example of predicting the memory usage of the server. First, we can use a self-made script to collect monitoring data within 24 hours, with a collection frequency of once every 10 seconds, for a total of 8640 data. The collected data includes the memory usage and timestamp of the server where the program is located. In this embodiment, each X includes 144 memory monitoring values, Y label The subsequent 48 memory monitoring values ​​of X are selected and divided into 49 and 10 training and test sets respectively. To ensure the experimental effect, the collection interval of some sample points in the dataset is not consistent with that of other sample points.

[0062] For this embodiment, the filter is constructed and calculated as follows:

[0063] Its sample matrix and label matrix are:

[0064]

[0065] The sample set X contains 49 samples, each of which contains 144 sample points with a fixed time sampling period, that is, the prediction window size of the recurrent neural network is 144, Y label are the labels of 49 samples.

[0066] For the above samples and labels, merge into the augmented matrix

[0067]

[0068] Set the filter size to 3*3, and the filter definition is:

[0069]

[0070]

[0071] Where, σ s , σ I , σ t The kernels are spatial, value, and time domains, respectively. G is a Gaussian function, W is a weight normalization term, S is the set of filter elements within the filter, j and i are elements within S, and j is the corresponding timestamp of the sample. This filter is based on matrix distance, sample value difference, and sample time difference, and consists of a spatial kernel, a value domain kernel, and a time domain kernel.

[0072] If the step size is set to 3 and padding is used, the sample set after filtering is:

[0073]

[0074] Compared with the original dataset, the sample size is 1 / 9 of the original matrix, which effectively realizes the downsampling operation.

[0075] In some embodiments of the present application, the downsampled datasets obtained by the methods provided herein can significantly improve the training effect of the model when training a recurrent neural network (GRU) compared to the existing technology. The following table shows the RMSE (root mean square error) corresponding to the training of recurrent neural networks using datasets obtained using different methods, which shows that the present application significantly outperforms the existing technology.

[0076]

[0077] By determining a first training data set; determining a target filter, wherein the target filter includes a spatial kernel, a value range kernel and a time domain kernel; filtering the first training data set based on the target filter to obtain a second training data set; and training an operation and maintenance indicator monitoring model based on the second training data set, by constructing a filter including a spatial kernel, a value range kernel and a time domain kernel to filter the training data set, the purpose of effectively reducing the fluctuating noise data in the training data set is achieved, thereby achieving the technical effect of improving the training effect of the operation and maintenance indicator monitoring model, and further solving the technical problem of poor model training effect caused by the inability to effectively reduce the adverse effects of fluctuating noise data on the model in the existing technology.

[0078] In addition, this application also provides a filter suitable for predicting IT operation and maintenance monitoring indicators under a recurrent neural network model, which can effectively reduce the adverse effects of fluctuating noise data on the model (denoising), while reducing the number of parameters during training (downsampling), and realize the prediction task of unequally spaced sampling samples, taking into account both prediction accuracy and speed.

[0079] In addition, the operation and maintenance indicator monitoring model training method provided by this application has the following beneficial effects compared to the prior art: it can effectively reduce sample noise and improve prediction accuracy. When the server is working, the frequent start and stop of process resources will cause fluctuations in monitoring indicators, which is equivalent to the impact of noise. The filter proposed in this patent can minimize the impact of noise while maintaining sample fluctuations and time characteristics. It can effectively downsample and improve prediction efficiency. Setting the step size of the filter to be greater than 1 can achieve downsampling operations while ensuring that the prediction accuracy is not lost too much. It can improve the prediction accuracy of the recurrent neural network model under non-equally spaced time samples, and introducing a time domain kernel in the filter can retain the differences in samples caused by changes in acquisition time.

[0080] According to an embodiment of the present invention, a method embodiment of a data filtering method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0081] Figure 2 is a data filtering method according to an embodiment of the present invention, such as Figure 2 As shown, the method includes the following steps:

[0082] Step S202, determining a training data set;

[0083] Step S204, determining a target filter, wherein the target filter includes a spatial kernel, a range kernel, and a time domain kernel;

[0084] Step S206, when the sample in the training data set is a first sample, denoising the first sample in the training data set based on the spatial kernel, wherein the first sample is a sample in the training data set whose sample value variance is less than a first preset variance threshold;

[0085] Step S208, when the sample in the training data set is a second sample, denoising the second sample in the training data set based on the range kernel, wherein the second sample is a sample in the training data set whose sample value variance is greater than a first preset variance threshold;

[0086] Step S210 , when the sample in the training data set is the third sample, denoising the third sample in the training data set based on the time domain kernel, wherein the third sample is a sample in the training data set whose sampling period variance is greater than the second preset variance threshold.

[0087] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0088] According to an embodiment of the present invention, a device embodiment of an operation and maintenance indicator monitoring model training device is provided. Figure 3 The invention provides an operation and maintenance plant protection monitoring model training device, such as Figure 3 As shown, the device includes: a first processing module 30, used to determine a first training data set; a second processing module 32, used to determine a target filter, wherein the target filter includes a spatial kernel, a value range kernel and a time domain kernel; a filtering module 34, used to filter the first training data set based on the target filter to obtain a second training data set; a training module 36, used to train the operation and maintenance indicator monitoring model based on the second training data set.

[0089] According to an embodiment of the present invention, an embodiment of a non-volatile storage medium is provided. The non-volatile storage medium includes a stored program, wherein, when the program is executed, the device containing the non-volatile storage medium is controlled to perform the following operation and maintenance indicator monitoring model training method: determining a first training data set; determining a target filter, wherein the target filter includes a spatial kernel, a range kernel, and a time domain kernel; filtering the first training data set based on the target filter to obtain a second training data set; and training the operation and maintenance indicator monitoring model based on the second training data set.

[0090] According to an embodiment of the present invention, an embodiment of an electronic device is provided. The electronic device includes a processor configured to run a program, wherein, when the program is run, the following operation and maintenance indicator monitoring model training method is performed: determining a first training data set; determining a target filter, wherein the target filter includes a spatial kernel, a range kernel, and a time-domain kernel; filtering the first training data set based on the target filter to obtain a second training data set; and training the operation and maintenance indicator monitoring model based on the second training data set.

[0091] According to an embodiment of the present invention, an embodiment of a computer terminal is also provided. Figure 4 FIG. 4 is a schematic structural diagram of a computer device 400 according to an embodiment of the present invention.

[0092] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by the processor 402 of the device 400 to complete the following operation and maintenance indicator monitoring model training method: determining a first training data set; determining a target filter, wherein the target filter includes a spatial kernel, a range kernel, and a time domain kernel; filtering the first training data set based on the target filter to obtain a second training data set; and training the operation and maintenance indicator monitoring model based on the second training data set. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0093] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0095] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0096] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0097] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0098] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for training an operation and maintenance indicator monitoring model, characterized in that: include: determining a first training data set; Determine a target filter, wherein the target filter includes a spatial kernel, a range kernel, and a time domain kernel, and the target filter is used to reduce noise on the first training data set. The spatial kernel is used to reduce noise on a first sample in the first training data set, wherein the first sample is a sample in the first training data set whose sample value variance is less than a first preset variance threshold; the range kernel is used to reduce noise on a second sample in the first training data set, wherein the second sample is a sample in the first training data set whose sample value variance is greater than a first preset variance threshold; and the time domain kernel is used to reduce noise on a third sample in the first training data set, wherein the third sample is a sample in the first training data set whose sampling period variance is greater than a second preset variance threshold; Filtering the first training data set based on the target filter to obtain a second training data set; The operation and maintenance indicator monitoring model is trained based on the second training data set.

2. The operation and maintenance indicator monitoring model training method according to claim 1, characterized in that: Determining a first training data set includes: Determining the size of the prediction window of the operation and maintenance indicator monitoring model; Determining a plurality of samples and a sampling period corresponding to each of the plurality of samples, wherein the sample is a server; Determining a sampling point corresponding to each of the multiple samples, wherein the number of sampling points corresponding to each sample is the same as the size of the prediction window, the sampling point including a timestamp indicating a sampling time point, and an indicator value of a target operation and maintenance indicator of the sample obtained by sampling at the sampling time point, where the target operation and maintenance indicator is an indicator monitored by the operation and maintenance indicator monitoring model; Determining a label corresponding to each sample in the plurality of samples; A sample set and a label set in the first training data set are determined, wherein the sample set includes a sampling point corresponding to each sample in the multiple samples, and the label set includes a label corresponding to each sample in the multiple samples.

3. The operation and maintenance indicator monitoring model training method according to claim 1 is characterized in that: Determining the target filter includes: Determine the spatial objective function, the range objective function, and the time domain objective function; Constructing a weight specification term of the target filter according to the spatial objective function, the range objective function, and the time domain objective function, wherein the spatial portion of the spatial objective function and the weight specification term components is the spatial kernel, the range portion of the range objective function and the weight specification term components is the range kernel, and the time domain portion of the time domain objective function and the weight specification term components is the time domain kernel; The target filter is constructed according to the weight specification term, the spatial objective function, the range objective function, and the time domain objective function, wherein the target filter is used to perform bilateral filtering on the first training data set.

4. The operation and maintenance indicator monitoring model training method according to claim 1, characterized in that: Filtering the training data set based on the target filter to obtain a second training data set includes: Determining a filtering step size of the target filter, wherein the filtering step size is greater than 1; The training data set is downsampled based on the target filter according to the filtering step size to obtain the second training data set.

5. The operation and maintenance indicator monitoring model training method according to claim 1, characterized in that: Filtering the first training data set based on the target filter to obtain a second training data set includes: Determine a sample set matrix corresponding to the sample set in the first training data set, and a label set matrix corresponding to the label set of the samples in the first training data set, wherein the elements in the sample set matrix are sampling points corresponding to each sample, the sampling point includes a timestamp indicating the sampling time point, and an indicator value of a target operation and maintenance indicator of the sample obtained by sampling at the sampling time point, the target operation and maintenance indicator is an indicator monitored by the operation and maintenance indicator monitoring model, and the elements in the label set matrix are labels corresponding to each sample; Merging the sample set matrix and the label set matrix into a first augmented matrix corresponding to the first training data set; The first augmented matrix is ​​filtered based on the target filter to obtain a second augmented matrix, and the second augmented matrix is ​​used as the second training data set.

6. A data filtering method, characterized in that: include: Determine the training dataset; Determine a target filter, wherein the target filter includes a spatial kernel, a range kernel, and a time domain kernel; When the sample in the training data set is a first sample, denoising the first sample in the training data set based on the spatial core, wherein the first sample is a sample in the training data set whose sample value variance is less than a first preset variance threshold; In a case where the sample in the training data set is a second sample, denoising the second sample in the training data set based on the value range, wherein the second sample is a sample in the training data set whose sample value variance is greater than a first preset variance threshold; When the sample in the training data set is a third sample, noise reduction is performed on the third sample in the training data set based on the time domain check, wherein the third sample is a sample in the training data set whose sampling period variance is greater than a second preset variance threshold.

7. A training device for an operation and maintenance indicator monitoring model, characterized in that: include: A first processing module is used to determine a first training data set; a second processing module, configured to determine a target filter, wherein the target filter includes a spatial kernel, a range kernel, and a time domain kernel; the target filter is configured to reduce noise on the first training data set; the spatial kernel is configured to reduce noise on a first sample in the first training data set, wherein the first sample is a sample in the first training data set whose sample value variance is less than a first preset variance threshold; the range kernel is configured to reduce noise on a second sample in the first training data set, wherein the second sample is a sample in the first training data set whose sample value variance is greater than a first preset variance threshold; and the time domain kernel is configured to reduce noise on a third sample in the first training data set, wherein the third sample is a sample in the first training data set whose sampling period variance is greater than a second preset variance threshold; a filtering module, configured to filter the first training data set based on the target filter to obtain a second training data set; A training module is used to train the operation and maintenance indicator monitoring model based on the second training data set.

8. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the operation and maintenance indicator monitoring model training method according to any one of claims 1 to 5.

9. An electronic device, characterized in that: The electronic device includes a processor, and the processor is used to run a program, wherein the program executes the operation and maintenance indicator monitoring model training method according to any one of claims 1 to 5 when running.

Images