Time series data monitoring method and device, electronic equipment and nonvolatile storage medium

Abstract

This application discloses a method, apparatus, electronic device, and non-volatile storage medium for monitoring time-series data. The method includes: determining a first temporal attention coefficient corresponding to the time-series data through a temporal convolutional subnetwork in a target network, and extracting features from the time-series data based on the first temporal attention coefficient to obtain a first target feature; extracting features from the time-series data through a Transformer subnetwork in the target network based on a second temporal attention coefficient to obtain a second target feature; determining a predicted value of the running data at a target time based on the first and second target features; and determining that the running data at the target time is abnormal if the deviation between the predicted value and the actual value of the running data at the target time exceeds a preset deviation threshold. This application solves the technical problem of poor accuracy in detecting anomalies in time-series data caused by the poor feature extraction effect of neural networks on time-series data in related technologies.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a time-series data monitoring method, apparatus, electronic device, and non-volatile storage medium. Background Technology

[0002] Temporal anomaly detection is a major application area in data mining. It involves extracting features from a target sequence to identify data objects whose expected predictions differ from the current data. Such data are called outliers. Temporal anomaly detection is widely used in many fields such as communications, power, and security.

[0003] Deep learning uses deep neural networks as its basic structure. Common networks used for temporal anomaly detection include CNNs, RNNs, and Transformers. However, these neural networks suffer from various problems when extracting features from temporal data. For example, the self-attention mechanism of Transformers often lacks focus on the most relevant information in the search region, resulting in poor performance in extracting local information; the ReLU function in fully convolutional neural networks (FCNs) prevents most components from being updated. These technical problems lead to poor accuracy in temporal anomaly detection.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a time-series data monitoring method, apparatus, electronic device, and non-volatile storage medium to at least solve the technical problem of poor accuracy in time-series data anomaly detection caused by the poor feature extraction effect of neural networks on time-series data in related technologies.

[0006] According to one aspect of the embodiments of this application, a time-series data monitoring method is provided, comprising: determining a first temporal attention coefficient corresponding to the time-series data through a temporal convolutional subnetwork in a target network, and extracting features from the time-series data based on the first temporal attention coefficient to obtain a first target feature, wherein the time-series data includes multiple runtime data recorded in chronological order, the runtime data being used to characterize the runtime status of a business system, and the first temporal attention coefficient being used to characterize the degree of mutual influence between runtime data at different times in the time-series data; determining a second temporal attention coefficient corresponding to the time-series data through a Transformer subnetwork in the target network, and extracting features from the time-series data based on the second temporal attention coefficient to obtain a second target feature, wherein the second temporal attention coefficient being used to characterize the importance of runtime data at different times in the time-series data; determining a predicted value of runtime data at a target time based on the first target feature and the second target feature, wherein the target time is the time immediately following the last time in the time-series data; and determining that the runtime data at the target time is abnormal if the deviation between the predicted value of the runtime data and the actual value of the runtime data at the target time exceeds a preset deviation threshold.

[0007] Optionally, the temporal convolutional subnetwork includes: a dilated causal convolutional layer, a weight normalization layer, an activation function layer, and a temporal attention module; feature extraction of temporal data based on the first temporal attention coefficient to obtain the first target feature includes: performing a convolution operation on the temporal data based on the dilated convolution kernel in the dilated causal convolutional layer to obtain the first feature sequence corresponding to the temporal data, wherein the weight values ​​in the dilated convolution kernel are normalized by the weight normalization layer; performing a nonlinear transformation on the first feature sequence based on the Gaussian error linear unit Gelu in the activation function layer to obtain the transformed first feature sequence; determining the first attention weights corresponding to each feature in the transformed first feature sequence based on the temporal attention module, and determining the first temporal attention coefficient based on the first attention weights; and filtering each feature in the first feature sequence based on the first temporal attention coefficient to obtain the first target feature.

[0008] Optionally, determining the second temporal attention coefficients corresponding to the temporal data through the Transformer subnetwork in the target network includes: vectorizing the temporal data and extracting features from the vectorized temporal data to obtain a second feature sequence; calculating the second feature sequence with the query matrix, key matrix, and value matrix respectively to obtain the query vector, key vector, and value vector; determining the target similarity between the query vector and the key vector, and determining the second attention weight based on the target similarity; and determining the second temporal attention coefficients based on the value vector and the second attention weight.

[0009] Optionally, determining the target similarity between the query vector and the key vector includes: mapping the query vector to a first binary code of a preset length using a hash function, and mapping the key vector to a second binary code of a preset length using a hash function; determining the Hamming distance between the first binary code and the second binary code, wherein the Hamming distance is used to characterize the degree of similarity between the first binary code and the second binary code; and determining the target similarity based on the Hamming distance.

[0010] Optionally, determining the predicted value of the running data at the target time based on the first target feature and the second target feature includes: determining the weight coefficients and bias coefficients corresponding to the first target feature and the second target feature; fusing the first target feature and the second target feature through the gated residual network in the target network based on the weight coefficients and bias coefficients to obtain the third target feature; and inputting the third target feature into the fully connected network in the target network for classification and prediction to obtain the predicted value of the running data at the target time.

[0011] Optionally, the target network is trained through the following steps: obtaining raw training data, which includes multiple historical running data recorded in chronological order; selecting N+1 historical running data points from the raw training data to obtain a training data sequence, where N is a positive integer; determining the default values ​​in the training data sequence and supplementing the default values ​​in the training data sequence based on the historical running data of the time immediately preceding and following the time immediately following the time corresponding to the default value; normalizing the training data sequence after supplementing the default values ​​to obtain the target training data; and training the initial network based on the target training data to obtain the target network.

[0012] Optionally, training the initial network based on the target training data to obtain the target network includes: using the initial network, predicting the historical running data corresponding to the N+1th time step based on the historical running data corresponding to the first N time steps in the target training data to obtain the predicted data value; determining the hyperparameters based on the predicted data value and the actual historical running data at the N+1th time step in the target training data, wherein the hyperparameters include at least one of the following: the learning rate of the network model, the number of iterations, and the number of network layers; adjusting the network parameters of the initial network based on the hyperparameters, and repeating the above steps until the performance parameters of the initial network meet the preset performance parameter threshold to obtain the target network.

[0013] According to another aspect of the embodiments of this application, a time-series data monitoring device is also provided, comprising: a first feature extraction module, configured to determine a first temporal attention coefficient corresponding to the time-series data through a temporal convolutional subnetwork in a target network, and to perform feature extraction on the time-series data based on the first temporal attention coefficient to obtain a first target feature, wherein the time-series data includes multiple running data recorded in chronological order, the running data being used to characterize the running state of a business system, and the first temporal attention coefficient being used to characterize the degree of mutual influence between running data at different times in the time-series data; a second feature extraction module, configured to determine a second temporal attention coefficient corresponding to the time-series data through a Transformer subnetwork in the target network, and to perform feature extraction on the time-series data based on the second temporal attention coefficient to obtain a second target feature, wherein the second temporal attention coefficient being used to characterize the importance of running data at different times in the time-series data; a data prediction module, configured to determine a predicted value of the running data at a target time based on the first target feature and the second target feature, wherein the target time is the time immediately following the last time in the time-series data; and an anomaly judgment module, configured to determine that the running data at the target time is abnormal when the deviation between the predicted value of the running data and the actual value of the running data at the target time exceeds a preset deviation threshold.

[0014] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a timing data monitoring method during runtime.

[0015] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes a timing data monitoring method by running the computer program.

[0016] In this embodiment, a first temporal attention coefficient corresponding to the temporal data is determined through a temporal convolutional subnetwork in the target network, and feature extraction is performed on the temporal data based on the first temporal attention coefficient to obtain a first target feature. The temporal data contains multiple runtime data recorded in chronological order, and the runtime data is used to characterize the operating state of the business system. The first temporal attention coefficient is used to characterize the degree of mutual influence between runtime data at different times in the temporal data. A second temporal attention coefficient corresponding to the temporal data is determined through a Transformer subnetwork in the target network, and feature extraction is performed on the temporal data based on the second temporal attention coefficient to obtain a second target feature. The second temporal attention coefficient is used to characterize the importance of runtime data at different times in the temporal data. Based on the first target feature and the second... The target feature determines the predicted value of the running data at the target time, where the target time is the time immediately following the last time in the time series data. If the deviation between the predicted value and the actual value of the running data at the target time exceeds a preset deviation threshold, an anomaly is determined in the running data at the target time. This is achieved by adding sparsity to the attention mechanism of the Transformer network to focus on the importance of data at different times, and by using Gaussian error linear units (Gelu) as the activation function in the FCN network. This enables the FCN network to better extract local information and invariant features, achieving accurate prediction of target values ​​in the field of time series data anomaly detection. This solves the technical problem of poor accuracy in time series data anomaly detection caused by the poor feature extraction performance of neural networks in related technologies. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for time-series data monitoring, according to an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of a method flow for time-series data monitoring according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of a target network structure provided according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the original attention mechanism in a Transformer according to an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of an improved attention mechanism in a Transformer according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of a model training process provided according to an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the structure of a time-series data monitoring device provided according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Temporal anomaly detection is a major application area in data mining. It involves extracting features from a target sequence to identify discrepancies between the network's predicted expectation and the actual data object. Such data are called outliers. Temporal anomaly detection is widely used in many fields, including communications, power, and security. For example, in intelligent operations and maintenance, temporal anomaly detection can ensure the secure, stable, and efficient operation of enterprise IT systems. It avoids system downtime caused by faults, reduces the risk to normal enterprise operations, and provides strong protection for enterprises. It also prevents damage to user experience.

[0028] Based on the current development of time-series anomaly detection, methods can be categorized into classical methods, machine learning-based methods, and deep learning-based methods. Classical methods mostly utilize Markov chain models, autoregressive models, and other similar techniques for anomaly detection. However, these methods are prone to overfitting with long-term sequences. Machine learning-based methods employ decision trees, vector regression, ridge regression, and other similar techniques. However, machine learning models may struggle to handle complex, non-linear data.

[0029] Deep learning uses deep neural networks as its basic structure. Common networks used for temporal anomaly detection include CNNs, RNNs, LSTMs, and Transformers. Transformers overcome the limitation of RNNs' inability to perform parallel operations and are more efficient than LSTMs. However, Transformers are less effective than CNNs at extracting local information. Therefore, CNNs can be used to extract local information and invariant features, while Transformers can be used to extract global and temporal information. Based on this, an EcoFormer-FCN method is proposed, which connects CNNs and Transformers in parallel. Multiple feature extractors are used to extract features from multivariate temporal data, fusing multi-view information to predict target values. However, the self-attention of Transformers often lacks focus on the most relevant information in the search region. Furthermore, the ReLU in FCN networks causes most components to never be updated, leading to technical problems in the accuracy of temporal anomaly detection.

[0030] To address the aforementioned issues, this application provides relevant solutions, which are detailed below.

[0031] According to an embodiment of this application, a method embodiment for time-series data monitoring is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or electronic device) for implementing a time-series data monitoring method is shown. Figure 1As shown, the computer terminal 10 (or electronic device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0033] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or electronic device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0034] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the timing data monitoring method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned timing data monitoring method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0035] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0036] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or electronic device).

[0037] Under the above operating environment, this application provides a method for time-series data monitoring. Figure 2 This is a schematic diagram of a time-series data monitoring method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0038] Step S202: Determine the first temporal attention coefficient corresponding to the temporal data through the temporal convolutional sub-network in the target network, and extract features from the temporal data based on the first temporal attention coefficient to obtain the first target feature. The temporal data contains multiple running data recorded in chronological order. The running data is used to characterize the running status of the business system, and the first temporal attention coefficient is used to characterize the degree of mutual influence between the running data at different times in the temporal data.

[0039] Figure 3 This is a schematic diagram of a target network structure provided according to an embodiment of this application, such as... Figure 3 As shown, the target network has two parallel branches: an improved TCN (i.e., the temporal convolutional sub-network) and an improved Transformer module (i.e., the Transformer sub-network). Features are input in parallel into these two parallel networks. The improved TCN network includes dilated causal convolutions, weight normalization, Gelu activation functions (i.e., the activation function layer), and a temporal attention module. The improved Transformer model extracts features through an improved attention mechanism and temporal attention. The extracted features are then input into a gated residual network. Effective information from the features is filtered, and the fused information is then input into a fully connected network to output the predicted target value.

[0040] The target network described above will be further explained below.

[0041] In some embodiments of this application, the temporal convolutional subnetwork includes: a dilated causal convolutional layer, a weight normalization layer, an activation function layer, and a temporal attention module; feature extraction of temporal data based on a first temporal attention coefficient to obtain a first target feature includes: performing a convolution operation on the temporal data based on the dilated convolution kernel in the dilated causal convolutional layer to obtain a first feature sequence corresponding to the temporal data, wherein the weight values ​​in the dilated convolution kernel are normalized by the weight normalization layer; performing a nonlinear transformation on the first feature sequence based on the Gaussian error linear unit Gelu in the activation function layer to obtain a transformed first feature sequence; determining the first attention weights corresponding to each feature in the transformed first feature sequence based on the temporal attention module, and determining the first temporal attention coefficient based on the first attention weights; and filtering each feature in the first feature sequence based on the first temporal attention coefficient to obtain the first target feature.

[0042] Specifically, the TCN branch (Temporal Convolutional Network), that is, the input of the aforementioned temporal convolutional sub-network, is X. int (i.e., the aforementioned time-series data), the output is feature F. tcn (i.e., the first target feature mentioned above).

[0043] Suppose we have an input sequence: x0, x0, ..., x T And we hope to predict some corresponding outputs: y0, y0, ..., y T When we predict y t When we consider the y-value at time t, we can only observe values ​​prior to time t.

[0044] The TCN network utilizes dilated convolutions to capture global information from the entire sequence and incorporates residual blocks.

[0045] Formula description of dilated convolution kernel: For a one-dimensional input sequence... and convolution kernel in, Let be an n-dimensional real vector space, and define the dilated convolution operation F on the sequence elements s as:

[0046]

[0047] Where d is the number of holes, k is the kernel size of the dilated convolution, and sd·i indicates which unit is used for computation.

[0048] Residual connections directly add the input features to the features after a series of transformations, as described by the following formula:

[0049] o = Activation(X + F(X))

[0050] Where X is the input feature, F(X) is a series of transformed features, and Activation() is the activation function.

[0051] In FCN networks, the ReLU activation function can prevent most components from being updated. In this embodiment, GeLU is used instead of the original ReLU and Dropout to increase the nonlinearity of the network model. This avoids the problem of most components being unable to be updated in the ReLU activation function, enabling the FCN network to better extract local information and invariant features.

[0052] Step S204: Determine the second temporal attention coefficient corresponding to the time series data through the Transformer sub-network in the target network, and extract features from the time series data based on the second temporal attention coefficient to obtain the second target feature. The second temporal attention coefficient is used to characterize the importance of the running data at different times in the time series data.

[0053] In some embodiments of this application, determining the second temporal attention coefficient corresponding to the temporal data through the Transformer subnetwork in the target network includes the following steps: vectorizing the temporal data and extracting features from the vectorized temporal data to obtain a second feature sequence; calculating the second feature sequence with the query matrix, key matrix, and value matrix respectively to obtain the query vector, key vector, and value vector; determining the target similarity between the query vector and the key vector, and determining the second attention weight based on the target similarity; and determining the second temporal attention coefficient based on the value vector and the second attention weight.

[0054] In some embodiments of this application, determining the target similarity between the query vector and the key vector includes the following steps: mapping the query vector to a first binary code of a preset length using a hash function, and mapping the key vector to a second binary code of a preset length using a hash function; determining the Hamming distance between the first binary code and the second binary code, wherein the Hamming distance is used to characterize the degree of similarity between the first binary code and the second binary code; and determining the target similarity based on the Hamming distance.

[0055] Specifically, the input to the Transformer branch, i.e., the Transformer sub-network mentioned above, is X. int (i.e., the aforementioned time-series data), the output is feature F. Transformer (i.e., the second target feature mentioned above).

[0056] In this implementation, a Transormer module is used, in which traditional self-attention mechanisms (such as...) are implemented. Figure 4 (as shown) is transformed into an EcoFormer attention mechanism (such as...) Figure 5 (As shown), to increase the sparsity of the network.

[0057] q represents the query vector, v represents the vector of the queried information (i.e., the value vector mentioned above), and k represents the vector of the relevance of the queried information to other information (i.e., the key vector mentioned above).

[0058] The kernel-based hash function maps qi and kj to b-bit (i.e., the preset length mentioned above) binary codes H(qi) (i.e., the first binary code mentioned above) and H(kj) (i.e., the second binary code mentioned above). The Hamming distance between them is shown below:

[0059]

[0060] Among them, H r (·) represents the r-th bit of the binary code. It is an indicator function that returns 1 if A is satisfied, and 0 otherwise.

[0061] The code inner product between H(qi) (i.e., the first binary code mentioned above) and H(kj) (i.e., the second binary code mentioned above) is shown in the following formula:

[0062]

[0063] The above equation demonstrates the equivalence between Hamming distance and code inner product. Because of the one-to-one correspondence, the improved EcoFormer attention can be represented as follows through hash lookup and key replacement:

[0064]

[0065] In this context, the subscripts i, j, and t of q, k, and v are used to represent different positions in the query vector, key vector, and value vector.

[0066] In related technologies, Transformer networks lack sparsity in their self-attention mechanisms due to a lack of focus on the most relevant information within a region. This application addresses this issue by using an improved Attention mechanism to increase sparsity, focusing on the importance of data at different time points, thus resolving the problem of self-attention's inability to focus on the most relevant information within a region in time-series data.

[0067] Step S206: Based on the first target feature and the second target feature, determine the predicted value of the running data at the target time, wherein the target time is the time immediately following the last time in the time series data;

[0068] In some embodiments of this application, determining the predicted value of the running data at the target time based on the first target feature and the second target feature includes the following steps: determining the weight coefficients and bias coefficients corresponding to the first target feature and the second target feature; fusing the first target feature and the second target feature through a gated residual network in the target network based on the weight coefficients and bias coefficients to obtain a third target feature; and inputting the third target feature into a fully connected network in the target network for classification and prediction to obtain the predicted value of the running data at the target time.

[0069] Specifically, the feature F obtained from the improved Transformer branch and the TCN branch is processed through a gated residual network. Transformer (i.e., the second target feature mentioned above) and F tcn The effective information of (i.e., the first target feature mentioned above) is filtered and fused, which can be done using the following formula:

[0070] GRN w (a,c)=LayerNorm(a+GLU w (η1))

[0071] η1=W 1,ω η2+b 1,ω

[0072] η2=ELU(W 2,ω a+W 3,ω c+b 2,ω )

[0073] In this system, the gated residual network (GRN) receives inputs a (first target feature) and c (second target feature), and ELU is an exponential linear unit activation function. and LayerNorm is the intermediate layer, ω is the exponent representing the shared weights, and GLU is the gated layer. and These represent the weighting coefficient and the bias coefficient, respectively. Represents a d model The real space of dimension d model The dimension of the embedded vectors in the model is defined. a+GLU w (η1) is a combination of linear and nonlinear functions to achieve the function of gated residual network.

[0074] This application integrates FCN and EcoFormer with a gated residual network after applying temporal attention. This allows for the fusion of features while filtering and discarding information, thus maximizing the retention of the impact of effective features on the final result.

[0075] Step S208: If the deviation between the predicted value of the running data and the actual value of the running data at the target time exceeds a preset deviation threshold, it is determined that the running data at the target time is abnormal.

[0076] The time-series data anomaly detection technology disclosed in this application can be applied to intelligent operation and maintenance scenarios to ensure the continuity and real-time operation of business systems, while also guaranteeing data reliability and security. It makes it easier for complex IT systems to handle operation and maintenance issues and allows for advance preparation for further tracking of anomaly information.

[0077] The training process for the target network described above will be further explained below.

[0078] This application acquires data and trains the network through the following steps: Temporal data is acquired, integrated, and preprocessed such as normalization is performed. Then, a temporal attention-based EcoFormer-FCN network (target network) is constructed, employing a gated convolutional network to fuse and filter information among multiple features. Appropriate hyperparameters, such as learning rate and momentum, are used to train and test the constructed network.

[0079] Figure 6 This is a schematic diagram of a model training process provided according to an embodiment of this application, such as... Figure 6 As shown, the process includes the following steps:

[0080] Step 1: Obtain time-series data (raw training data) and preprocess the data;

[0081] In some embodiments of this application, the target network is trained through the following steps: obtaining original training data, wherein the original training data contains multiple historical running data recorded in chronological order; selecting N+1 historical running data points from the original training data to obtain a training data sequence, wherein N is a positive integer; determining the default value in the training data sequence, and supplementing the default value in the training data sequence based on the historical running data of the time immediately preceding the time corresponding to the default value, and the historical running data of the time immediately following the time corresponding to the default value; performing a normalization operation on the training data sequence after supplementing the default value to obtain target training data; and training the initial network based on the target training data to obtain the target network.

[0082] Specifically, the collected N+1 time-series data (i.e., the training data sequence mentioned above) are processed. The first N data points are selected and slid along the time dimension according to a time window T, and prediction is performed using a model. The historical running data Y = (y1, y2, ..., y...) from the first N time points is used. T To predict the target at time T+1.

[0083] For missing data in the training data sequence, linear interpolation is used to fill in the gaps. This is done by fitting the trend changes in the data. For example, univariate linear regression can be used.

[0084] To simplify model computation, a data normalization method can also be used. Assume the j-th value of the time series sequence (the training data sequence after supplementing with missing values) is... Normalization is performed using a formula. The values ​​are normalized to the range of 0-1, where maxi represents the maximum value in the sequence and mini represents the minimum value in the sequence;

[0085] Step 2: Construct the initial time-attention-based EcoFormer-FCN network (i.e., the initial network);

[0086] Step 3: Use appropriate hyperparameters to train and test the constructed model.

[0087] In some embodiments of this application, training an initial network based on target training data to obtain a target network includes the following steps: using the initial network, predicting the historical running data corresponding to the N+1th time step based on the historical running data corresponding to the first N time steps in the target training data to obtain a predicted data value; determining hyperparameters based on the predicted data value and the actual historical running data at the N+1th time step in the target training data, wherein the hyperparameters include at least one of the following: the learning rate of the network model, the number of iterations, and the number of network layers; adjusting the network parameters of the initial network based on the hyperparameters, and repeating the above steps until the performance parameters of the initial network meet the preset performance parameter threshold to obtain the target network.

[0088] Specifically, select relevant hyperparameters, such as the number of iterations and the learning rate, to train the model using the training set. Use the mean absolute error loss as the loss function, and train the network using backpropagation and gradient descent (SGD). Store the trained network. There are n values ​​in total, predicted. i Represents the i-th predicted value, actual i Let the i-th true value be represented by the mean absolute error loss function:

[0089]

[0090] This application improves the FCN network and adds temporal attention to extract features, enhancing the interaction and importance of different time points in time-series data. It also fuses features between the EcoFormer network and the improved FCN network in parallel, maximizing the preservation of effective features' impact on the final result. The sparse attention method addresses the lack of focus on the most relevant information in regions under time-series data conditions caused by self-attention. Optimizations have been made to address the issue of most components in the ReLU activation function failing to be updated. This achieves accurate prediction of target values ​​in the field of time-series data anomaly detection, providing guidance for the safe, stable, and efficient operation of intelligent maintenance.

[0091] Through the above steps, by adding sparsity to the attention mechanism of the Transformer network to focus on the importance of data at different times, and by using Gaussian error linear units (Gelu) as activation functions in the FCN network, the FCN network can better extract local information and invariant features, thereby achieving the goal of accurately predicting target values ​​in the field of time series data anomaly detection. This solves the technical problem of poor accuracy in time series data anomaly detection caused by the poor feature extraction effect of neural networks on time series data in related technologies.

[0092] According to an embodiment of this application, an embodiment of a time-series data monitoring device is also provided. Figure 7 This is a schematic diagram of a time-series data monitoring device according to an embodiment of this application. Figure 7 As shown, the device includes:

[0093] The first feature extraction module 70 is used to determine the first time attention coefficient corresponding to the time series data through the temporal convolutional sub-network in the target network, and to extract features from the time series data based on the first time attention coefficient to obtain the first target feature. The time series data contains multiple running data recorded in chronological order. The running data is used to characterize the running status of the business system, and the first time attention coefficient is used to characterize the degree of mutual influence between the running data at different times in the time series data.

[0094] The second feature extraction module 72 is used to determine the second time attention coefficient corresponding to the time series data through the Transformer sub-network in the target network, and to extract features from the time series data based on the second time attention coefficient to obtain the second target feature. The second time attention coefficient is used to characterize the importance of the running data at different times in the time series data.

[0095] The data prediction module 74 is used to determine the predicted value of the running data at the target time based on the first target feature and the second target feature, wherein the target time is the time immediately following the last time in the time series data;

[0096] The anomaly detection module 76 is used to determine that there is an anomaly in the running data at the target time when the deviation between the predicted value of the running data and the actual value of the running data at the target time exceeds a preset deviation threshold.

[0097] It should be noted that each module in the aforementioned timing data monitoring device can be a program module (e.g., a set of program instructions to implement a specific function) or a hardware module. For the latter, it can take the following forms, but is not limited to them: each of the above modules is represented by a processor, or the functions of each of the above modules are implemented by a processor.

[0098] It should be noted that the time-series data monitoring device provided in this embodiment can be used to perform... Figure 2 The time series data monitoring method shown above is also applicable to the embodiments of this application, and will not be repeated here.

[0099] This application embodiment also provides a non-volatile storage medium, which includes a stored computer program. The device where the non-volatile storage medium is located executes the following time-series data monitoring method by running the computer program: determining the first time attention coefficient corresponding to the time-series data through the time-series convolutional subnetwork in the target network, and extracting features from the time-series data based on the first time attention coefficient to obtain the first target feature. The time-series data includes multiple running data recorded in chronological order. The running data is used to characterize the running status of the business system, and the first time attention coefficient is used to characterize the degree of mutual influence between the running data at different times in the time-series data. The second temporal attention coefficient corresponding to the time series data is determined through the Transformer sub-network in the target network, and the time series data is feature extracted based on the second temporal attention coefficient to obtain the second target feature. The second temporal attention coefficient is used to characterize the importance of the running data at different times in the time series data. Based on the first target feature and the second target feature, the predicted value of the running data at the target time is determined, where the target time is the time immediately following the last time in the time series data. If the deviation between the predicted value of the running data and the actual value of the running data at the target time exceeds a preset deviation threshold, it is determined that the running data at the target time is abnormal.

[0100] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0101] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0106] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A time-series data monitoring method, characterized in that, include: The first temporal attention coefficient corresponding to the temporal data is determined through the temporal convolutional sub-network in the target network, and the features of the temporal data are extracted based on the first temporal attention coefficient to obtain the first target feature. The temporal data contains multiple running data recorded in chronological order. The running data is used to characterize the running status of the business system, and the first temporal attention coefficient is used to characterize the degree of mutual influence between the running data at different times in the temporal data. The temporal convolutional subnetwork includes: a dilated causal convolutional layer, a weight normalization layer, an activation function layer, and a temporal attention module. Feature extraction of the temporal data based on the first temporal attention coefficient to obtain the first target feature includes: performing a convolution operation on the temporal data using the dilated convolutional kernel in the dilated causal convolutional layer to obtain a first feature sequence corresponding to the temporal data, wherein the weight values ​​in the dilated convolutional kernel are normalized by the weight normalization layer; performing a nonlinear transformation on the first feature sequence using the Gaussian error linear unit (Gelu) in the activation function layer to obtain a transformed first feature sequence; determining the first attention weights corresponding to each feature in the transformed first feature sequence based on the temporal attention module, and determining the first temporal attention coefficient based on the first attention weights; and filtering each feature in the first feature sequence based on the first temporal attention coefficient to obtain the first target feature. The second temporal attention coefficient corresponding to the time series data is determined through the Transformer sub-network in the target network, and the second target feature is obtained by extracting features from the time series data based on the second temporal attention coefficient. The second temporal attention coefficient is used to characterize the importance of the running data at different times in the time series data. The determination of the second temporal attention coefficient corresponding to the time-series data through the Transformer sub-network in the target network includes: vectorizing the time-series data and extracting features from the vectorized time-series data to obtain a second feature sequence; calculating the second feature sequence with the query matrix, key matrix, and value matrix respectively to obtain a query vector, key vector, and value vector; determining the target similarity between the query vector and the key vector, including: mapping the query vector to a first binary code of a preset length using a hash function, and mapping the key vector to a second binary code of the preset length using the hash function; determining the Hamming distance between the first binary code and the second binary code, wherein the Hamming distance is used to characterize the similarity between the first binary code and the second binary code; determining the target similarity based on the Hamming distance; determining the second attention weight based on the target similarity; and determining the second temporal attention coefficient based on the value vector and the second attention weight. Based on the first target feature and the second target feature, the predicted value of the running data at the target time is determined, wherein the target time is the next time immediately following the last time in the time series data; If the deviation between the predicted value of the running data and the actual value of the running data at the target time exceeds a preset deviation threshold, it is determined that the running data at the target time is abnormal.

2. The time-series data monitoring method according to claim 1, characterized in that, Based on the first target feature and the second target feature, the predicted value of the running data at the target time is determined as follows: Determine the weight coefficients and bias coefficients corresponding to the first target feature and the second target feature; The first target feature and the second target feature are fused using the gated residual network in the target network based on the weight coefficients and the bias coefficients to obtain the third target feature; The third target feature is input into the fully connected network in the target network for classification and prediction to obtain the predicted value of the running data at the target time.

3. The time-series data monitoring method according to claim 1, characterized in that, The target network is trained through the following steps: Obtain the raw training data, wherein the raw training data contains multiple historical running data recorded in chronological order; N+1 time points of historical running data are selected from the original training data to obtain a training data sequence, where N is a positive integer; Determine the default value in the training data sequence, and supplement the default value in the training data sequence based on the historical running data of the time immediately preceding the time corresponding to the default value and the historical running data of the time immediately following the time corresponding to the default value. The training data sequence after adding the missing values ​​is normalized to obtain the target training data; The initial network is trained based on the target training data to obtain the target network.

4. The time-series data monitoring method according to claim 3, characterized in that, The initial network is trained based on the target training data to obtain the target network, which includes: Using the initial network, based on the historical running data corresponding to the first N time steps in the target training data, the historical running data corresponding to the N+1 time step is predicted to obtain the predicted data value. Based on the predicted data values ​​and the actual historical running data at the (N+1)th time step in the target training data, the hyperparameters are determined, wherein the hyperparameters include at least one of the following: the learning rate of the network model, the number of iterations, and the number of network layers; The network parameters of the initial network are adjusted according to the hyperparameters, and the above steps are repeated until the performance parameters of the initial network meet the preset performance parameter threshold, thus obtaining the target network.

5. A time-series data monitoring device, characterized in that, include: The first feature extraction module is used to determine the first temporal attention coefficient corresponding to the temporal data through the temporal convolutional sub-network in the target network, and to extract features from the temporal data based on the first temporal attention coefficient to obtain the first target feature. The temporal data contains multiple running data recorded in chronological order. The running data is used to characterize the running status of the business system. The first temporal attention coefficient is used to characterize the degree of mutual influence between the running data at different times in the temporal data. The temporal convolutional subnetwork includes: a dilated causal convolutional layer, a weight normalization layer, an activation function layer, and a temporal attention module. Feature extraction of the temporal data based on the first temporal attention coefficient to obtain the first target feature includes: performing a convolution operation on the temporal data using the dilated convolutional kernel in the dilated causal convolutional layer to obtain a first feature sequence corresponding to the temporal data, wherein the weight values ​​in the dilated convolutional kernel are normalized by the weight normalization layer; performing a nonlinear transformation on the first feature sequence using the Gaussian error linear unit (Gelu) in the activation function layer to obtain a transformed first feature sequence; determining the first attention weights corresponding to each feature in the transformed first feature sequence based on the temporal attention module, and determining the first temporal attention coefficient based on the first attention weights; and filtering each feature in the first feature sequence based on the first temporal attention coefficient to obtain the first target feature. The second feature extraction module is used to determine the second temporal attention coefficient corresponding to the time series data through the Transformer sub-network in the target network, and to extract features from the time series data based on the second temporal attention coefficient to obtain the second target feature, wherein the second temporal attention coefficient is used to characterize the importance of the running data at different times in the time series data; The determination of the second temporal attention coefficient corresponding to the time-series data through the Transformer sub-network in the target network includes: vectorizing the time-series data and extracting features from the vectorized time-series data to obtain a second feature sequence; calculating the second feature sequence with the query matrix, key matrix, and value matrix respectively to obtain a query vector, key vector, and value vector; determining the target similarity between the query vector and the key vector, including: mapping the query vector to a first binary code of a preset length using a hash function, and mapping the key vector to a second binary code of the preset length using the hash function; determining the Hamming distance between the first binary code and the second binary code, wherein the Hamming distance is used to characterize the similarity between the first binary code and the second binary code; determining the target similarity based on the Hamming distance; determining the second attention weight based on the target similarity; and determining the second temporal attention coefficient based on the value vector and the second attention weight. The data prediction module is used to determine the predicted value of the running data at a target time based on the first target feature and the second target feature, wherein the target time is the next time immediately following the last time in the time series data; The anomaly detection module is used to determine that the running data at the target time is abnormal when the deviation between the predicted value of the running data and the actual value of the running data at the target time exceeds a preset deviation threshold.

6. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the timing data monitoring method according to any one of claims 1 to 4.

7. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the timing data monitoring method according to any one of claims 1 to 4 by running the computer program.

Images