Operation and maintenance time series data processing method and device, storage medium and electronic equipment

Abstract

The present disclosure provides a method, device, storage medium and electronic device for processing operation and maintenance time series data, and relates to the field of computer technology. The method includes: acquiring operation and maintenance time series data, preprocessing the operation and maintenance time series data, and obtaining preprocessed operation and maintenance time series data; decomposing the preprocessed operation and maintenance time series data, and extracting feature sequence data from the preprocessed operation and maintenance time series data; calculating the feature sequence data according to a preset initial period, and obtaining a candidate period set of the operation and maintenance time series data; searching in the candidate period set, and determining the optimal period set of the operation and maintenance time series data. The method can accurately identify the periodicity of the operation and maintenance time series data, and thus can ensure the accuracy of threshold-free monitoring.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for processing operation and maintenance time series data, a storage medium, and an electronic device. Background Art

[0002] Whether it is the business system or the network, database, middleware, storage and other components, they are all important objects that need to be guaranteed by operation and maintenance. Real-time monitoring of the fluctuation of operation and maintenance time series data helps to ensure the stable and effective operation of the system. Considering that the operation and maintenance time series data has periodicity, the periodicity of the operation and maintenance time series data can be analyzed, and then the periodicity can be used to monitor the operation and maintenance time series data without threshold.

[0003] In the related technology, frequency domain analysis and time domain analysis can be used to obtain the periodicity of operation and maintenance time series data. However, the frequency domain analysis method and the time domain analysis method have the problem of inaccurate period prediction, which is easily disturbed by noise and outliers and cannot meet the accuracy requirements of non-threshold monitoring of operation and maintenance time series data.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a method, device, storage medium and electronic device for processing operation and maintenance time series data to solve the problem of inaccurate period prediction existing in frequency domain analysis method and time domain analysis method.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, there is provided an operation and maintenance time series data processing, including: acquiring operation and maintenance time series data, preprocessing the operation and maintenance time series data, and obtaining the preprocessed operation and maintenance time series data; decomposing the preprocessed operation and maintenance time series data, and extracting feature sequence data from the preprocessed operation and maintenance time series data; calculating the feature sequence data according to a preset initial period, and obtaining a candidate period set of the operation and maintenance time series data; searching in the candidate period set, and determining an optimal period set of the operation and maintenance time series data.

[0008] In an optional embodiment of the present disclosure, the number of the initial cycles is at least one; and the calculating the feature sequence data according to the preset initial cycle to obtain the candidate cycle set of the operation and maintenance time series data includes: using a Fourier transform algorithm to obtain the target sequence data corresponding to the feature sequence data; for each initial cycle, calculating the autocorrelation function score of the target sequence data to obtain the autocorrelation function score corresponding to each initial cycle; selecting an initial cycle whose autocorrelation function score is not less than a preset autocorrelation function score threshold, and generating the candidate cycle set using the selected initial cycle.

[0009] In an optional embodiment of the present disclosure, the use of a Fourier transform algorithm to obtain target sequence data corresponding to the feature sequence data includes: performing a Fourier transform on the feature sequence data to obtain a first Fourier transform result corresponding to the feature sequence data; extracting the first Fourier transform result based on a preset number of Fourier cycles to obtain a second Fourier transform result; and performing an inverse Fourier transform on the second Fourier transform result to obtain the target sequence data.

[0010] In an optional embodiment of the present disclosure, searching in the candidate cycle set to determine the optimal cycle set for the operation and maintenance time series data includes: verifying the candidate cycles in the candidate cycle set to determine the candidate cycles that pass the verification as target cycles, so as to generate a target cycle set using the target cycles; searching in the target cycle set by an optimization method to determine the optimal cycle set for the operation and maintenance time series data.

[0011] In an optional embodiment of the present disclosure, the candidate cycles in the candidate cycle set are verified to determine that the candidate cycle that passes the verification is the target cycle, including: segmenting the feature sequence data according to the candidate cycle to obtain at least one subsequence corresponding to the candidate cycle; counting the number of peaks contained in each subsequence in the at least one subsequence; based on a shape distance algorithm, calculating the shape distance between each subsequence and a related subsequence of each subsequence, wherein the time delay between the related subsequence of each subsequence and each subsequence is an integer multiple of a long period; verifying the candidate cycle according to the counted number of peaks and the calculated shape distance; if the verification passes, determining that the candidate cycle is the target cycle, and obtaining the periodic characteristics of the target cycle.

[0012] In an optional embodiment of the present disclosure, searching in the target cycle set by an optimization method to determine the optimal cycle set of the operation and maintenance time series data includes: determining the objective function to be the root mean square error; searching in the target cycle set by the L-BFGS optimization method to determine the search result with the smallest root mean square error value as the optimal cycle set.

[0013] In an optional embodiment of the present disclosure, the preprocessing of the operation and maintenance time series data includes: performing burr value identification and missing value supplementation processing, normalization processing and detrending processing on the operation and maintenance time series data.

[0014] In an optional embodiment of the present disclosure, the operation and maintenance time series data includes data of at least two long periods.

[0015] According to another aspect of the present disclosure, there is provided an operation and maintenance time series data processing device, comprising: a data preprocessing module, used to acquire operation and maintenance time series data, preprocess the operation and maintenance time series data, and obtain the preprocessed operation and maintenance time series data; a data extraction module, used to decompose the preprocessed operation and maintenance time series data, and extract feature sequence data from the preprocessed operation and maintenance time series data; a candidate cycle generation module, used to calculate the feature sequence data according to a preset initial cycle, and obtain a candidate cycle set of the operation and maintenance time series data; an optimal cycle determination module, used to search in the candidate cycle set, and determine the optimal cycle set of the operation and maintenance time series data.

[0016] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned operation and maintenance time series data processing method is implemented.

[0017] According to another aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned operation and maintenance time series data processing method by executing the executable instructions.

[0018] The operation and maintenance time series data processing method provided by the embodiments of the present disclosure, on the one hand, pre-processes the operation and maintenance time series data and then decomposes it to extract feature sequence data, which can be subsequently analyzed to determine the optimal period set, and can filter out interference data in the operation and maintenance time series data, reduce the amount of calculation, and improve the accuracy of the determined optimal period set; on the other hand, the feature sequence data is calculated using a preset initial period, so that a candidate period set can be obtained from the initial period, and then further search is performed in the candidate period set to obtain the optimal period set, which solves the problem of inaccurate period prediction existing in frequency domain analysis methods and time domain analysis methods, can accurately identify the periodicity of the operation and maintenance time series data, is not interfered by noise and outliers, and can thus ensure the accuracy of threshold-free monitoring.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 is a schematic diagram of an exemplary system architecture for processing operation and maintenance time series data according to an exemplary embodiment;

[0022] Figure 2 is a flow chart of a method for processing operation and maintenance time series data according to an exemplary embodiment;

[0023] Figure 3 is a schematic diagram of operation and maintenance time series data according to an exemplary embodiment;

[0024] Figure 4 is a schematic diagram of decomposition of operation and maintenance time series data according to an exemplary embodiment;

[0025] Figure 5 is a flowchart of a method for generating a candidate cycle set according to an exemplary embodiment;

[0026] Figure 6 is a schematic diagram of operation and maintenance time series data according to yet another exemplary embodiment;

[0027] Figure 7 is a flowchart of a method for processing operation and maintenance time series data according to another exemplary embodiment;

[0028] Figure 8 is a block diagram of an operation and maintenance time series data processing device 800 according to an exemplary embodiment; and

[0029] Fig. 9 The present invention is a structural block diagram of an electronic device for processing operation and maintenance time series data according to an exemplary embodiment. DETAILED DESCRIPTION

[0030] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0032] Figure 1 FIG. 1 is a schematic diagram of an exemplary system architecture for processing operation and maintenance time series data according to an exemplary embodiment. Figure 1 As shown, the system architecture may include a server 101, a network 102, and a client 103. The network 102 is used to provide a medium for a communication link between the client 103 and the server 101. The network 102 may include various connection types, such as wired, wireless communication links, or optical fiber cables.

[0033] In some optional embodiments, the client 103 for data transmission with the server 101 may include but is not limited to electronic devices of the type of smart phones, desktop computers, tablet computers, laptop computers, smart speakers, digital assistants, AR (Augmented Reality) devices, VR (Virtual Reality) devices, smart wearable devices, etc., or the client 103 may also be a personal computer, such as a laptop computer and a desktop computer, etc. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux, Windows, etc.

[0034] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some practical applications, server 101 can also be a server of a network platform, which can be, for example, a trading platform, a live broadcast platform, a social platform, or a music platform, etc., which is not limited in the embodiments of the present disclosure. Among them, the server can be a single server or a cluster formed by multiple servers, and the present disclosure does not limit the specific architecture of the server.

[0035] In some optional embodiments, the process of server 101 for implementing the operation and maintenance time series data processing method may be: server 101 obtains operation and maintenance time series data, preprocesses the operation and maintenance time series data, and obtains preprocessed operation and maintenance time series data; server 101 decomposes the preprocessed operation and maintenance time series data, and extracts feature sequence data from the preprocessed operation and maintenance time series data; server 101 calculates the feature sequence data according to a preset initial period to obtain a candidate period set of the operation and maintenance time series data; server 101 searches in the candidate period set to determine the optimal period set of the operation and maintenance time series data.

[0036] In addition, it should be noted that Figure 1 What is shown is only one application environment of the operation and maintenance time series data processing method provided by the present disclosure. Figure 1 The numbers of clients, networks and servers in the embodiment are merely illustrative, and any number of clients, networks and servers may be provided according to actual needs.

[0037] In order to enable ordinary persons in the art to better understand the technical solution of the present disclosure, each step of the operation and maintenance time series data processing method in the exemplary embodiment of the present disclosure will be described in more detail below with reference to the accompanying drawings and embodiments.

[0038] Figure 2 is a flowchart of a method for processing operation and maintenance time series data according to an exemplary embodiment. Figure 2 The execution subject of the method provided in the embodiment may be any electronic device, such as Figure 1 The server 101 in the embodiment, but the present disclosure is not limited thereto.

[0039] like Figure 2 As shown, the operation and maintenance time series data processing method provided by the embodiment of the present disclosure may include the following steps S210 to S240.

[0040] Step S210: Acquire operation and maintenance time series data, preprocess the operation and maintenance time series data, and obtain preprocessed operation and maintenance time series data.

[0041] Operation and maintenance time series data can be understood as monitoring indicator data in the field of operation and maintenance, such as basic resource monitoring indicators such as host monitoring indicators, database monitoring indicators, cache monitoring indicators, and application-side monitoring indicators, as well as business monitoring indicators such as transaction volume, success rate, response rate, and time consumption. Operation and maintenance time series data is data collected at different times. This type of data is collected in chronological order and is used for situations where the described phenomenon changes over time. In addition, the fluctuation of operation and maintenance time series data can reflect the operation of the business. Most businesses are cyclical, and the corresponding operation and maintenance time series data has a regular fluctuation pattern. According to the type of fluctuation anomaly, it can be divided into: burr-type anomaly, context anomaly, cliff / steep slope anomaly, climbing anomaly, amplitude anomaly, wave frequency anomaly, amplitude + wave frequency anomaly. It can be seen that real-time monitoring of the fluctuation request of operation and maintenance time series data can timely detect the type of anomaly, which helps to ensure the stable and effective operation of the system.

[0042] Operation and maintenance time series data has periodicity, but there is a problem of inconsistency between long and short periods. Periodic time series data is usually composed of one or more time series components with different periods, that is, multi-period time series. In other words, operation and maintenance time series data can have long periods and short periods. Each long-period operation and maintenance time series data can be composed of one or more time series components with different short periods. Figure 3 FIG. 1 is a schematic diagram showing operation and maintenance time series data according to an exemplary embodiment. Figure 3 It can be seen that the long cycle of the operation and maintenance time series data is 3 days, and the short cycles are 1 day and 2 days. This means that every 3 days of data constitutes a long cycle, and each long cycle of time series data is composed of short cycles of 1 day and 2 days. Figure 3 It can also be seen that the data from days 1 to 3 and days 4 to 6 have long cycle characteristics and satisfy the long cycle of 3 days; the data from day 1 and the data from day 4 have the characteristics of a short cycle of 1 day, that is, the data from day 1 and day 4 are the same or similar; the data from days 2 to 3 and the data from days 5 to 6 have the characteristics of a short cycle of 2 days, that is, the data from days 2 to 3 and the data from days 5 to 6 are the same or similar.

[0043] In addition, since each long-period operation and maintenance time series data can be composed of one or more different short-period time series components, the data processing method provided by the embodiment of the present disclosure can analyze the operation and maintenance time series data and identify its periodicity, that is, determine the long and short periods corresponding to the operation and maintenance time series data. Figure 3 It can be seen that the short-cycle characteristics can be analyzed from at least two long-cycle data, so in the embodiment of the present disclosure, the operation and maintenance time series data may include at least two long-cycle data.

[0044] In the operation and maintenance time series data processing method provided by the embodiment of the present disclosure, the operation and maintenance time series data can be analyzed to obtain the periodicity of the operation and maintenance time series data, and then the periodicity can be used to perform threshold-free monitoring on the operation and maintenance time series data. Among them, threshold-free monitoring means that there is no fixed threshold, and the normal monitoring value at the current time point is predicted based on the historical accumulated experience. Threshold-free monitoring can be automatically adjusted with the changes in system structure and function, and the risks it brings to the production system are completely controllable. Time series prediction, abnormal probability and risk alarm can be performed through threshold-free monitoring. Among them, time series prediction refers to establishing a monitoring item prediction model based on the historical data of monitoring values ​​to predict the current monitoring value; abnormal probability refers to establishing a distribution model based on the difference historical data to determine whether the current monitoring value is abnormal; risk alarm refers to judging whether the current monitoring item needs to be warned based on the historical data of continuous abnormal accumulation. Threshold-free monitoring of operation and maintenance time series data can be understood as: learning the data changes of operation and maintenance time series data by machine learning, and adaptively learning the indicator threshold of operation and maintenance time series data.

[0045] In an optional embodiment of the present disclosure, preprocessing the operation and maintenance time series data may include: performing burr value identification and missing value supplementation processing, normalization processing, and detrending processing on the operation and maintenance time series data.

[0046] In the data preprocessing stage, the main tasks are to identify burr values, supplement missing values, normalize, and detrend the operation and maintenance time series data to obtain smooth operation and maintenance time series data. The burr value can be understood as the peak / trough of the fluctuation of the time series data within a certain interval, which is significantly different from the value of the stable part of the data. Among them, the burr value identification method can be the local interval peak and trough identification and the global burr diagnosis based on the local peak and trough identification results. That is, if the peak / trough appears periodically, it is not an identified burr value, which belongs to the normal fluctuation phenomenon of the data; if the peak / trough does not appear periodically, it is identified as a burr point. After the burr value is identified and removed, the replacement of the burr value belongs to the missing value supplement problem. The specific missing value supplement method is a filling method based on the weighted calculation of the second-order exponential smoothing of year-on-year and month-on-month data, that is, the main idea is to jointly predict the non-abnormal value of the current point based on the true value and the predicted value at the same period position of the burr value point. Among them, the predicted value is obtained by the second-order exponential smoothing method, and the true value is obtained by the normally distributed random number constructed by the remaining points in the same period.

[0047] In addition, the normalization processing, the detrending processing, and the spur value identification and missing value supplementation processing may be performed in the following order: spur value identification and missing value supplementation processing, the normalization processing, and the detrending processing.

[0048] In the embodiments of the present disclosure, normalization processing, detrending processing, burr value identification and missing value supplementation processing can be used to pre-process the operation and maintenance time series data, so as to obtain smooth operation and maintenance time series data. Interference data in the operation and maintenance time series data can also be filtered out to avoid the impact of interference data in the operation and maintenance time series data, reduce the amount of calculation, and improve the accuracy of the determined optimal period set.

[0049] Step S220: decomposing the preprocessed operation and maintenance time series data, and extracting feature sequence data from the preprocessed operation and maintenance time series data.

[0050] Feature sequence data can be understood as an important component of operation and maintenance time series data, which can reflect the periodicity of operation and maintenance time series data. Figure 4 FIG. 1 is a schematic diagram showing the decomposition of operation and maintenance time series data according to an exemplary embodiment. Figure 4 As shown in Figure 2, the operation and maintenance time series data can be decomposed into trend component, periodic component and residual component. Figure 4 The periodic component in can be regarded as the characteristic sequence data obtained by decomposition.

[0051] Specifically, the pre-processed operation and maintenance time series data can be decomposed by discrete wavelet transform to obtain high-frequency and low-frequency components in the data, and then the sequence components that interfere with the determination of periodic characteristics are eliminated to obtain characteristic sequence data in the pre-processed operation and maintenance time series data. In the disclosed embodiment, the characteristic sequence data is extracted by discrete wavelet transform, and the characteristic sequence data can be subsequently analyzed to determine the optimal period set of the operation and maintenance time series data, thereby avoiding excessive calculation of components with small weights and insignificant to the overall results, improving calculation efficiency, and avoiding the influence of insignificant components on the results, thereby improving the accuracy of the optimal period set.

[0052] Step S230: Calculate the feature sequence data according to a preset initial period to obtain a candidate period set of the operation and maintenance time series data.

[0053] Among them, the initial period is a pre-set period, and the characteristic sequence data can be calculated according to the preset initial period, and then the candidate period is extracted from the initial period according to the calculation result to obtain the candidate period set. In order to ensure the accuracy of the candidate period, and then ensure the accuracy of the optimal period set determined subsequently, the initial period can be set according to the actual business situation, or the initial period can be assumed a priori. For example, the initial period is 3 minutes, 5 minutes, 1 hour, 3 hours, 6 hours, 12 hours, 1 day, 2 days, 3 days, 4 days, 5 days, 6 days, and 7 days.

[0054] In an optional embodiment of the present disclosure, calculating the feature sequence data according to a preset initial period to obtain a candidate period set of operation and maintenance time series data may include: using a Fourier transform algorithm to obtain target sequence data corresponding to the feature sequence data; for each initial period, calculating the autocorrelation function score of the target sequence data to obtain the autocorrelation function score corresponding to each initial period; selecting an initial period whose autocorrelation function score is not less than a preset autocorrelation function score threshold, and generating a candidate period set using the selected initial period.

[0055] Among them, the Fourier transform can express a function that meets certain conditions as a linear combination of trigonometric functions (sine and / or cosine functions) or their integrals. In the process of obtaining a set of candidate periods, the feature sequence data can be first subjected to Fourier transform analysis. Further, using the Fourier transform algorithm to obtain the target sequence data corresponding to the feature sequence data can include: performing a Fourier transform on the feature sequence data to obtain a first Fourier transform result corresponding to the feature sequence data; extracting the first Fourier transform result based on a preset number of Fourier periods to obtain a second Fourier transform result; and performing an inverse Fourier transform on the second Fourier transform result to obtain the target sequence data.

[0056] The principle of Fourier transform is that any continuously measured time series or signal can be expressed as an infinite superposition of sinusoidal signals of different frequencies. First, the characteristic sequence data is Fourier transformed to obtain the first Fourier transform result. The characteristic sequence data can be expressed as the superposition of sinusoidal signals of different frequencies, so the first Fourier transform result can be regarded as the superposition of sinusoidal signals at different frequencies. Then, the first Fourier transform result is extracted by the preset number of Fourier cycles to obtain the second Fourier transform result. Among them, the preset number of Fourier cycles is equivalent to the preset number of main frequency components, which is set to k. In other words, the obtained sinusoidal signals at different frequencies are sorted from high to low according to the frequency, and the k components with high frequency are selected to obtain the second Fourier transform result. Then, the second Fourier transform result is inversely Fourier transformed to obtain the target sequence data.

[0057] After obtaining the target sequence data through Fourier transform, the autocorrelation function score of the target sequence data corresponding to each initial period can be calculated. Among them, the autocorrelation function describes the dependence of a random signal at one moment and another moment, that is, the correlation between two random variables at time t1 and time t2 is studied. In the embodiment of the present disclosure, for each initial period, the autocorrelation function score of the target sequence data corresponding to the initial period is calculated. The closer the autocorrelation function score is to 1, the greater the possibility that the operation and maintenance time series data contains the time series component of the initial period. Specifically, the autocorrelation function score threshold can be pre-set, and the initial period whose autocorrelation function score is not less than the autocorrelation function score threshold can be selected, and the candidate period set is generated by the selected initial period. For example, the autocorrelation function score threshold is pre-set to 0.6, and when the initial period is 1 day, the corresponding autocorrelation function score is calculated to be 0.7, then 1 day is considered to be a candidate period, and when the initial period is 0.5 days, the corresponding autocorrelation function score is calculated to be 0.5, then 0.5 days is considered not to be a candidate period.

[0058] Figure 5 FIG. 1 is a flow chart of a method for generating a candidate cycle set according to an exemplary embodiment. Figure 5 As shown, the method for generating a candidate cycle set may include:

[0059] Step S510: Perform Fourier transform on the characteristic sequence data to obtain a first Fourier transform result corresponding to the characteristic sequence data;

[0060] Step S520: extracting the first Fourier transform result based on a preset number of Fourier cycles to obtain a second Fourier transform result;

[0061] Step S530: performing inverse Fourier transform on the second Fourier transform result to obtain target sequence data;

[0062] Step S540: randomly selecting an initial period;

[0063] Step S550: For the selected initial period, calculate the autocorrelation function score of the target sequence data to obtain the autocorrelation function score corresponding to the initial period;

[0064] Step S560: determining whether the autocorrelation function score corresponding to the initial period is greater than or equal to a preset autocorrelation function score threshold, if so, executing step S570, if not, executing step S580;

[0065] Step S570: Determine the initial period as a candidate period;

[0066] Step S580: Determine whether each initial cycle has been analyzed, if so, execute step S590, if not, execute step S540;

[0067] Step S590: Generate a candidate cycle set using the candidate cycles.

[0068] In the disclosed embodiment, the characteristic sequence data is first analyzed by Fourier transform to extract the target sequence data, and then the autocorrelation function score of the target sequence data corresponding to each initial period is calculated, and then the autocorrelation function score corresponding to each calculated initial period is judged by a preset autocorrelation function score threshold, and the candidate period is selected, and then a candidate period set can be generated. On the one hand, the target sequence data is extracted by Fourier transform, and the target sequence data is subsequently analyzed to obtain the candidate period, which reduces the amount of calculation, can reduce the influence of noise and outliers, improve the accuracy of the generated candidate period set, and further improve the accuracy of the optimal period set; on the other hand, after the Fourier transform analysis, the candidate period set is generated by the autocorrelation function, and the analysis is performed by combining the time domain and the frequency domain, which can further improve the accuracy of the generated candidate period set.

[0069] Step S240: Search in the candidate cycle set to determine the optimal cycle set for the operation and maintenance time series data.

[0070] Furthermore, searching in the candidate cycle set to determine the optimal cycle set for the operation and maintenance time series data may include: verifying the candidate cycles in the candidate cycle set, determining the candidate cycles that pass the verification as the target cycles, and generating a target cycle set using the target cycles; searching in the target cycle set by an optimization method to determine the optimal cycle set for the operation and maintenance time series data.

[0071] After obtaining the candidate cycle set, each candidate cycle in the candidate cycle set can be verified, and the candidate cycle that passes the verification can be determined as the target cycle, so that the target cycle set can be generated. Then, the optimal cycle set of the operation and maintenance time series data is obtained by searching in the target cycle set through the optimization method.

[0072] In an optional embodiment of the present disclosure, verifying the candidate cycles in the candidate cycle set and determining that the candidate cycle that passes the verification is the target cycle can include: segmenting the feature sequence data according to the candidate cycle to obtain at least one subsequence corresponding to the candidate cycle; counting the number of peaks contained in each subsequence in at least one subsequence; calculating the shape distance between each subsequence and the related subsequence of each subsequence based on the shape distance algorithm, wherein the time delay between the related subsequence of each subsequence and each subsequence is an integer multiple of the long period; verifying the candidate cycle based on the counted number of peaks and the calculated shape distance; if the verification passes, determining that the candidate cycle is the target cycle, and obtaining the periodic characteristics of the target cycle.

[0073] For ease of explanation, the following is an explanation of the verification of a candidate period T. First, use the candidate period T to segment the feature sequence data to obtain at least one subsequence corresponding to the candidate period T. For example, the operation and maintenance time series data is data from the 1st day to the 14th day, and the long period is 7 days. The operation and maintenance time series data is preprocessed and then decomposed to obtain the feature sequence data from the 1st day to the 14th day. If the candidate period T is 2 days, the feature sequence data from the 1st to the 7th day is divided into 7 subsequences, namely subsequence a1{data from the 1st day, data from the 2nd day}, subsequence a2{data from the 2nd day, data from the 3rd day}, subsequence a3{data from the 3rd day, data from the 4th day}, subsequence a4{data from the 4th day, data from the 5th day}, subsequence a5{data from the 5th day, data from the 6th day}, subsequence a6{data from the 6th day, data from the 7th day}, subsequence a7{data from the 7th day, data from the 1st day}, The characteristic sequence data from the 8th day to the 14th day are divided into 7 subsequences, namely subsequence b1{data from the 8th day, data from the 9th day}, subsequence b2{data from the 9th day, data from the 10th day}, subsequence b3{data from the 10th day, data from the 11th day}, subsequence b4{data from the 11th day, data from the 12th day}, subsequence b5{data from the 12th day, data from the 13th day}, subsequence b6{data from the 13th day, data from the 14th day}, and subsequence b7{data from the 14th day, data from the 8th day}.

[0074] After obtaining the subsequences corresponding to the candidate period T, the number of peaks contained in each subsequence can be counted. And, based on the shape distance algorithm, the shape distance between each subsequence and the related subsequence of each subsequence is calculated. The shape distance algorithm is called the shape-based distance algorithm, that is, the SBD algorithm, which is one of the methods for measuring distances in time series and can be used to calculate the similarity between sequences. Specifically, for the time series X(x1, x2, x3, …, x n ) and time series Y(y1,y2,y3,……,y n ), there is a time delay s between the two sequences, and the SBD distance algorithm for calculating the similarity of the two curves is as follows:

[0075]

[0076]

[0077] SBD(X,Y)=1-NCC(X,Y)

[0078] Among them, NCC is the standardized mutual correlation coefficient of sequences X and Y. The value range of NCC is between -1 and 1, which is similar to the Pearson correlation coefficient. The final calculated SBD algorithm result is between 0 and 2. The closer to 0, the stronger the correlation between sequences X and Y. When SBD is 0, it means that the fluctuation curves of sequences X and Y are consistent and they are the same sequence.

[0079] In the embodiment of the present disclosure, the shape distance between each subsequence and the related subsequence of the subsequence is calculated. The time delay between the subsequence and the related subsequence of the subsequence is an integer multiple of the long period, that is, the subsequences whose time delay is an integer multiple of the long period are correlated with each other. Continuing with the above example, the related subsequence of subsequence a1 is subsequence b1, the related subsequence of subsequence a2 is subsequence b2, the related subsequence of subsequence a3 is subsequence b3, the related subsequence of subsequence a4 is subsequence b4, the related subsequence of subsequence a5 is subsequence b5, the related subsequence of subsequence a6 is subsequence b6, and the related subsequence of subsequence a7 is subsequence b7.

[0080] Next, the candidate period T can be verified based on the statistical number of peaks and the calculated shape distance. Specifically, after calculating the shape distance between each subsequence and the subsequence, a shape distance matrix can be generated. Select the minimum value in the shape distance matrix to obtain the subsequence corresponding to the minimum value. Then, determine whether the number of peaks contained in the subsequence is the same as the number of peaks contained in the related subsequence of the subsequence. If they are the same, the candidate period T is considered to have been verified. If they are not the same, the candidate period T can be considered to have failed the verification.

[0081] Continuing with the above example, the shape distance between subsequence a1 and subsequence b1, the shape distance between subsequence a2 and subsequence b2, the shape distance between subsequence a3 and subsequence b3, the shape distance between subsequence a4 and subsequence b4, the shape distance between subsequence a5 and subsequence b5, the shape distance between subsequence a6 and subsequence b6, and the shape distance between subsequence a7 and subsequence b7 are calculated. Analysis shows that the shape distance between subsequence a1 and subsequence b1 is the smallest. Then, determine whether the number of peaks contained in subsequence a1 is the same as the number of peaks contained in subsequence b1. If they are the same, it is considered that candidate cycle 2 has passed the verification.

[0082] In addition, the following method can also be used to verify the candidate period T. For a subsequence corresponding to the candidate period T, if the number of peaks contained in the subsequence is the same as the number of peaks contained in the related subsequence of the subsequence, and the shape distance between the subsequence and the related subsequence of the subsequence is less than the preset distance threshold, the candidate period T can be considered to have been verified. If the number of peaks contained in each subsequence corresponding to the candidate period T is different from the number of peaks contained in the related subsequence of each subsequence, it can be considered that the candidate period T has not been verified. If the shape distance between each subsequence corresponding to the candidate period T and the related subsequence of each subsequence is not less than the preset distance threshold, it can be considered that the candidate period T has not been verified. Continuing with the above example, if the number of peaks contained in subsequence a1 is the same as the number of peaks contained in subsequence b1, and the shape distance between subsequence a1 and the related subsequence b1 of subsequence a1 is less than the preset distance threshold, it can be considered that the candidate period 2 has been verified.

[0083] In addition, if the verification is successful, the candidate period can be determined to be the target period, and the periodic characteristics of the target period can be obtained. Through the periodic characteristics, it can be determined which data in a long period of data meets the target period. Continuing with the above example, if candidate period 2 is verified, it can be obtained that the periodic characteristics of the data on the 1st and 2nd days meet period 2, that is, the data on the 1st and 2nd days of a long period of 7 days can constitute a time series component with a short period of 2.

[0084] It should be noted that the above verification of the candidate cycles in the candidate cycle set is described by taking one candidate cycle as an example. The embodiment of the present disclosure can verify each candidate cycle in the candidate cycle set and generate a target cycle set using all the candidate cycles that have passed the verification.

[0085] In the disclosed embodiment, after obtaining the candidate cycle set, each candidate cycle in the candidate cycle set is verified based on the peak value check and SBD algorithm. The candidate cycle can be verified from two aspects: the number of peaks and the shape similarity. The candidate cycles that fail the verification are discarded, thereby further improving the accuracy of the optimal cycle set.

[0086] In an optional embodiment of the present disclosure, searching in the target cycle set by an optimization method to determine the optimal cycle set of operation and maintenance time series data may include: determining the objective function to be the root mean square error; searching in the target cycle set by the L-BFGS optimization method to determine the search result with the smallest root mean square error value as the optimal cycle set.

[0087] Among them, the L-BFGS algorithm is an improved BFGS algorithm, where BFGS is the abbreviation of the initials of the four inventors Broyden, Fletcher, Goldfarb and Shanno. The L-BFGS optimization method is a common method for solving unconstrained nonlinear programming problems, with the advantages of fast convergence speed and low memory overhead.

[0088] After obtaining the target period set, the objective function can be determined to be the root mean square error. The L-BFGS optimization method is used to search in the target period set, and the search result that minimizes the root mean square error value is determined to be the optimal period set. For example, the target period set is {1,2,5,7}. Since the target period set {1,2,5,7} contains multiple subsets, the root mean square error corresponding to each subset can be calculated. If the subset is {1}, the operation and maintenance time series data of the previous 7 days is used to predict the operation and maintenance time series data of the 8th day, and then the root mean square error between the predicted value and the true value of the data on the 8th day is calculated. The objective function is the root mean square error. The solution that minimizes the root mean square error value of the objective function by searching with the L-BFGS optimization method is the optimal period set, which can speed up the overall model calculation, quickly obtain the optimal solution, and quickly obtain the optimal period set.

[0089] In the disclosed embodiment, the candidate cycles in the candidate cycle set are verified based on the peak value test and the SBD algorithm to obtain the target cycle set. Then, the root mean square error is used as the objective function, and the L-BFGS optimization method is used to search in the target cycle set to obtain the optimal cycle set of the operation and maintenance time series data. In this way, the obtained optimal cycle set can be passed into the threshold-free monitoring model, and then the operation and maintenance time series data can be monitored. In addition, according to the historical data of the operation and maintenance time series, the most valuable cycle memory weight in the historical data can be increased to make the threshold-free monitoring more accurate.

[0090] For example, the optimal period set {2,5,7} is obtained. Figure 6 FIG. 1 is a schematic diagram showing operation and maintenance time series data according to another exemplary embodiment. Figure 6 As shown in the figure, the long cycle of the operation and maintenance time series data is 7 days, and the short cycles are 2 days and 5 days. The operation and maintenance time series data to be predicted belongs to the first short cycle, that is, the 2-day cycle. The following methods can be used to correct the prediction results:

[0091] result1=f(input=[T1,T2,T3,T4,T5,T6,T7])

[0092] result2=f(input=[T1,T2])

[0093] final_result=(result1*a+result2*b)

[0094] Among them, f represents the baseline model used to learn the historical performance of indicator data, Ti represents the data of the i-th day, and a and b represent weight coefficients. result1 represents the result obtained by long-term prediction, result2 represents the result obtained by the first short-term prediction, and final_result represents the final prediction result obtained by weighted summation of the long-term prediction results and the short-term prediction results. In the training phase, the true value of the most recent period is retained as the evaluation target, and the solution of a and b that makes the final baseline prediction effect optimal is searched. The values ​​of a and b will be updated in time according to the changes in the indicator data. It can be seen that a and b are equivalent to the introduced correction terms, which can give more weight to the most valuable memory cycle in the historical data, so that the threshold-free baseline model fitting effect is optimal, and the threshold-free baseline monitoring of the operation and maintenance time series data is more accurately performed, and possible abnormal points are detected in real time.

[0095] Figure 7 FIG. 1 is a flowchart of a method for processing operation and maintenance time series data according to another exemplary embodiment. Figure 7 As shown in the figure, the main process of the operation and maintenance time series data processing method can include: first preprocessing the acquired operation and maintenance time series data, such as burr value processing, normalization processing, and detrending processing; then, using discrete wavelet transform to decompose the preprocessed data and extract feature sequence data; then, using the preset initial period, calculating the autocorrelation function of the feature sequence data to obtain a candidate period set; then, based on the peak test and SBD algorithm, verifying the candidate periods in the candidate period set to obtain a target period set; finally, using the root mean square error as the objective function, searching in the target period set through the L-BFGS optimization method to obtain the optimal period set of the operation and maintenance time series data, so that the obtained optimal period set can be used for threshold-free monitoring.

[0096] Figure 8 8 is a block diagram of an operation and maintenance time series data processing device 800 according to an exemplary embodiment. Figure 8 The device includes: a data preprocessing module 810, a data extraction module 820, a candidate cycle generation module 830 and an optimal cycle determination module 840.

[0097] Among them, the data preprocessing module 810 can be used to: obtain operation and maintenance time series data, preprocess the operation and maintenance time series data, and obtain preprocessed operation and maintenance time series data; the data extraction module 820 can be used to: decompose the preprocessed operation and maintenance time series data, and extract feature sequence data from the preprocessed operation and maintenance time series data; the candidate cycle generation module 830 can be used to: calculate the feature sequence data according to a preset initial cycle to obtain a candidate cycle set of operation and maintenance time series data; the optimal cycle determination module 840 can be used to: search in the candidate cycle set to determine the optimal cycle set of operation and maintenance time series data.

[0098] In an optional embodiment of the present disclosure, the number of initial cycles is at least one. And, the candidate cycle generation module 830 can also be used to: use the Fourier transform algorithm to obtain the target sequence data corresponding to the feature sequence data; for each initial cycle, calculate the autocorrelation function score of the target sequence data to obtain the autocorrelation function score corresponding to each initial cycle; select the initial cycle whose autocorrelation function score is not less than a preset autocorrelation function score threshold, and use the selected initial cycle to generate a candidate cycle set.

[0099] In an optional embodiment of the present disclosure, the candidate period generation module 830 may also be used to: perform Fourier transform on the feature sequence data to obtain a first Fourier transform result corresponding to the feature sequence data; extract the first Fourier transform result based on a preset number of Fourier periods to obtain a second Fourier transform result; and perform an inverse Fourier transform on the second Fourier transform result to obtain target sequence data.

[0100] In an optional embodiment of the present disclosure, the optimal cycle determination module 840 can also be used to: verify the candidate cycles in the candidate cycle set, determine the candidate cycles that have passed the verification as the target cycles, and use the target cycles to generate a target cycle set; search in the target cycle set through an optimization method to determine the optimal cycle set of operation and maintenance time series data.

[0101] In an optional embodiment of the present disclosure, the optimal period determination module 840 may also be used to: divide the feature sequence data according to the candidate period to obtain at least one subsequence corresponding to the candidate period; count the number of peaks contained in each subsequence in at least one subsequence; calculate the shape distance between each subsequence and the related subsequence of each subsequence based on the shape distance algorithm, wherein the time delay between the related subsequence of each subsequence and each subsequence is an integer multiple of the long period; verify the candidate period according to the counted number of peaks and the calculated shape distance; if the verification passes, determine that the candidate period is the target period, and obtain the periodic characteristics of the target period.

[0102] In an optional embodiment of the present disclosure, the optimal period determination module 840 may also be used to: determine the objective function as the root mean square error; search in the target period set by the L-BFGS optimization method, and determine the search result with the smallest root mean square error value as the optimal period set.

[0103] In an optional embodiment of the present disclosure, the data preprocessing module 810 may also be used to: perform burr value identification and missing value supplementation processing, normalization processing, and detrending processing on the operation and maintenance time series data.

[0104] In an optional embodiment of the present disclosure, the operation and maintenance time series data may include data of at least two long periods.

[0105] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0106] Fig. 9 is a structural block diagram of an electronic device for processing operation and maintenance time series data according to an exemplary embodiment. Fig. 9 hereinafter describes an electronic device 900 according to such an embodiment of the present disclosure. Fig. 9 The electronic device 900 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0107] like Fig. 9 As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include, but are not limited to: the at least one processing unit 910, the at least one storage unit 920, a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910), and a display unit 940.

[0108] The storage unit stores program codes, which can be executed by the processing unit 910, so that the processing unit 910 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 910 can perform the following steps: Figure 2 Follow the steps shown in .

[0109] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 921 and / or a cache memory unit 922 , and may further include a read-only memory unit (ROM) 923 .

[0110] The storage unit 920 may also include a program / utility 924 having a set (at least one) of program modules 925, such program modules 925 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0111] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0112] The electronic device 900 may also communicate with one or more external devices 970 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 950. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 960. As shown, the network adapter 960 communicates with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0113] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0114] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to complete the above method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0115] In an exemplary embodiment, a computer program product is also provided, including a computer program and / or instructions, and when the computer program and / or instructions are executed by a processor, the online file processing method in the above embodiment is implemented.

[0116] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0117] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for processing operation and maintenance time series data, characterized in that: include: Acquire operation and maintenance time series data, preprocess the operation and maintenance time series data, and obtain the preprocessed operation and maintenance time series data; Decomposing the preprocessed operation and maintenance time series data, and extracting characteristic sequence data from the preprocessed operation and maintenance time series data; The characteristic sequence data is the periodic component in the preprocessed operation and maintenance time series data; Calculating the characteristic sequence data according to a preset initial period to obtain a candidate period set of the operation and maintenance time series data; searching in the candidate period set to determine an optimal period set of the operation and maintenance time series data; The searching in the candidate cycle set to determine the optimal cycle set of the operation and maintenance time series data includes: verifying the candidate cycles in the candidate cycle set, determining the candidate cycles that pass the verification as the target cycles, and using the target cycles to generate the target cycle set; searching in the target cycle set by an optimization method to determine the optimal cycle set of the operation and maintenance time series data; The method of searching in the target cycle set by an optimization method to determine the optimal cycle set of the operation and maintenance time series data includes: determining the objective function to be a root mean square error; searching in multiple subsets included in the target cycle set by an L-BFGS optimization method to determine the search result with the smallest root mean square error value corresponding to the subset as the optimal cycle set; the root mean square error corresponding to the subset is calculated based on the predicted value corresponding to the subset and the true value corresponding to the predicted value.

2. The method according to claim 1, characterized in that: The number of the initial cycles is at least one; and, The step of calculating the characteristic sequence data according to a preset initial period to obtain a candidate period set of the operation and maintenance time series data includes: Using a Fourier transform algorithm, obtaining target sequence data corresponding to the characteristic sequence data; For each initial period, calculating the autocorrelation function score of the target sequence data to obtain the autocorrelation function score corresponding to each initial period; An initial period whose autocorrelation function score is not less than a preset autocorrelation function score threshold is selected, and the candidate period set is generated using the selected initial period.

3. The method according to claim 2, characterized in that The method of using a Fourier transform algorithm to obtain target sequence data corresponding to the characteristic sequence data includes: Performing Fourier transform on the characteristic sequence data to obtain a first Fourier transform result corresponding to the characteristic sequence data; Extracting the first Fourier transform result based on a preset number of Fourier cycles to obtain a second Fourier transform result; Perform an inverse Fourier transform on the second Fourier transform result to obtain the target sequence data.

4. The method according to claim 1, characterized in that: The verifying the candidate cycles in the candidate cycle set and determining the candidate cycle that passes the verification as the target cycle includes: Segment the feature sequence data according to the candidate period to obtain at least one subsequence corresponding to the candidate period; Counting the number of peaks contained in each subsequence of the at least one subsequence; Calculating a shape distance between each subsequence and a related subsequence of each subsequence based on a shape distance algorithm, wherein a time delay between the related subsequence of each subsequence and each subsequence is an integer multiple of a long period; Verifying the candidate period according to the counted number of peaks and the calculated shape distance; If the verification passes, the candidate period is determined to be the target period, and the period characteristic of the target period is obtained.

5. The method according to claim 1, characterized in that: The preprocessing of the operation and maintenance time series data includes: The operation and maintenance time series data is subjected to burr value identification, missing value supplementation, normalization and detrending processing.

6. The method according to any one of claims 1 to 5, characterized in that: The operation and maintenance time series data includes data of at least two long periods.

7. An operation and maintenance time series data processing device, characterized in that: include: A data preprocessing module, used to obtain operation and maintenance time series data, preprocess the operation and maintenance time series data, and obtain the preprocessed operation and maintenance time series data; A data extraction module, used to decompose the preprocessed operation and maintenance time series data and extract characteristic sequence data from the preprocessed operation and maintenance time series data; The characteristic sequence data is the periodic component in the preprocessed operation and maintenance time series data; A candidate cycle generation module is used to calculate the characteristic sequence data according to a preset initial cycle to obtain a candidate cycle set of the operation and maintenance time series data; An optimal cycle determination module, used to search in the candidate cycle set to determine the optimal cycle set of the operation and maintenance time series data; The optimal cycle determination module is further used to verify the candidate cycles in the candidate cycle set, determine the candidate cycles that pass the verification as the target cycles, and use the target cycles to generate a target cycle set; search in the target cycle set by an optimization method to determine the optimal cycle set of the operation and maintenance time series data; The optimal period determination module is also used to determine that the objective function is a root mean square error; search in multiple subsets contained in the target period set through the L-BFGS optimization method, and determine that the search result with the smallest root mean square error value corresponding to the subset is the optimal period set; the root mean square error corresponding to the subset is calculated based on the predicted value corresponding to the subset and the true value corresponding to the predicted value.

8. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the operation and maintenance time series data processing method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the operation and maintenance time series data processing method as described in any one of claims 1 to 6.

Images