Multi-dimensional time sequence anomaly detection method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a multi-dimensional time sequence anomaly detection method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining a multi-dimensional time sequence, and carrying out the masking of the multi-dimensional time sequence, and obtaining a masked sequence; performing feature extraction on the multi-dimensional time sequence based on a time self-attention mechanism to obtain a first sequence feature; performing feature extraction on the multi-dimensional time sequence based on a feature self-attention mechanism to obtain a second sequence feature; reconstructing the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence; determining an anomaly score through an anomaly detection model; and judging the abnormal score based on a preset threshold value, and determining that the multidimensional time sequence is abnormal when the abnormal score is greater than or equal to the preset threshold value. According to the method and the device, the sequence dependency relationship in a single time sequence can be accurately captured, the interdependence between time sequences of different dimensions is informed, and the accuracy and the stability of anomaly detection are improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data analysis, and in particular to a multi-dimensional time series anomaly detection method, device, electronic device, storage medium and program product. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] Time series anomaly detection is an important data analysis technique used to identify and mark abnormal or unconventional data points and sequences that appear in a series of time-ordered data; these data consist of values ​​indexed by consecutive timestamps, each of which corresponds to a specific time position; they record the progression of different phenomena over time; compared with image and text data, time series data has obvious temporal properties and continuity, and can show the trends and laws of things over time.

[0004] However, related technologies rely too much on the intuitive features of data, but fail to deeply explore and utilize the deep-level characteristics of data, and thus fail to identify more complex abnormal patterns that are not based on surface features. Summary of the invention

[0005] In view of this, the purpose of the present disclosure is to propose a multidimensional time series anomaly detection method, device, electronic device, storage medium and program product, which at least to a certain extent solve one of the technical problems in the related art.

[0006] Based on the above purpose, the first aspect of the exemplary embodiment of the present disclosure provides a multi-dimensional time series anomaly detection method, which is applied to a server, and the method includes:

[0007] Determine a multidimensional time series, and mask the multidimensional time series to obtain a masked sequence;

[0008] Extracting features from the multidimensional time series based on a temporal self-attention mechanism to obtain first sequence features;

[0009] Extracting features from the multidimensional time series based on a feature self-attention mechanism to obtain second sequence features;

[0010] Reconstructing the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence;

[0011] Determine, by a pre-trained anomaly detection model, an error between the multidimensional time series and the reconstructed sequence as an anomaly score;

[0012] The abnormality score is judged based on a preset threshold, and in response to the abnormality score being greater than or equal to the preset threshold, it is determined that the multidimensional time series has an abnormality.

[0013] Based on the same inventive concept, the second aspect of the exemplary embodiment of the present disclosure provides a multi-dimensional time series anomaly detection device, comprising:

[0014] a mask sequence determination module, configured to determine a multidimensional time series, mask the multidimensional time series, and obtain a masked sequence;

[0015] A first feature determination module is configured to extract features from the multidimensional time series based on a temporal self-attention mechanism to obtain a first sequence feature;

[0016] A second feature determination module is configured to extract features from the multidimensional time series based on a feature self-attention mechanism to obtain a second sequence feature;

[0017] a reconstructed sequence determining module, configured to reconstruct the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence;

[0018] an anomaly score determination module, configured to determine an error between the multidimensional time series and the reconstructed sequence as an anomaly score through a pre-trained anomaly detection model;

[0019] The detection result determination module is configured to judge the abnormality score based on a preset threshold, and determine that an abnormality exists in the multidimensional time series in response to the abnormality score being greater than or equal to the preset threshold.

[0020] Based on the same inventive concept, a third aspect of the exemplary embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0021] Based on the same inventive concept, a fourth aspect of the exemplary embodiments of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in the first aspect.

[0022] Based on the same inventive concept, a fifth aspect of the exemplary embodiments of the present disclosure provides a computer program product, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes the method described in the first aspect.

[0023] From the above, it can be seen that the multidimensional time series anomaly detection method, device, electronic device, storage medium and program product provided by the embodiment of the present disclosure include: determining a multidimensional time series, masking the multidimensional time series to obtain a masked sequence; extracting features from the multidimensional time series based on a time self-attention mechanism to obtain a first sequence feature; extracting features from the multidimensional time series based on a feature self-attention mechanism to obtain a second sequence feature; reconstructing the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence; determining the error between the multidimensional time series and the reconstructed sequence as an anomaly score through a pre-trained anomaly detection model; judging the anomaly score based on a preset threshold, and determining that the multidimensional time series has an anomaly in response to the anomaly score being greater than or equal to the preset threshold. The present disclosure can not only accurately capture the sequential dependencies in a single time series, but also gain insight into the interdependencies between time series of different dimensions, thereby improving the accuracy and stability of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A schematic diagram of an application scenario of a multidimensional time series anomaly detection method provided by an exemplary embodiment of the present disclosure;

[0026] Figure 2 A schematic diagram of a flow chart of a multi-dimensional time series anomaly detection method provided by an exemplary embodiment of the present disclosure;

[0027] Figure 3 A schematic diagram of a structure of a multi-dimensional time series anomaly detection device provided by an exemplary embodiment of the present disclosure;

[0028] Figure 4 A schematic diagram of the hardware structure of an electronic device provided for an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, usage scenarios, etc. of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0030] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present application according to the prompt message.

[0031] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0032] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of the present application. Other methods that meet the relevant laws and regulations may also be applied to the implementation method of the present application.

[0033] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0034] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, the principles and spirit of the present disclosure will be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0035] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0036] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. The article "one" or "a" before an element does not exclude the existence of multiple such elements.

[0037] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

[0038] As described in the background technology, the related technologies rely too much on the intuitive features of the data, but fail to deeply explore and utilize the deep-level characteristics of the data, and thus fail to identify more complex abnormal patterns that are not based on surface features. Specifically, time series anomaly detection is a crucial data analysis method, which aims to identify data points or subsequences that deviate from the normal pattern from time series information. This type of data consists of a series of observations recorded in chronological order, and each observation usually corresponds to a specific time point. These data can depict the changing process of physical phenomena, such as sensor readings of industrial equipment. Unlike image and text data, time series data exhibit strong temporal order and coherence, which reveal the dynamic trends and internal laws of things over time. For example, equipment sensor data can reflect the operating status of the equipment.

[0039] The unique properties of time series data make it have significant application value in many fields. In terms of industrial equipment maintenance, analyzing the operation data of the equipment can predict potential failures in advance, achieve preventive maintenance, and reduce maintenance costs. For example, monitoring the sensor data such as vibration and temperature of the equipment can help identify possible failure points in advance and avoid interruptions in the production line. In the manufacturing industry, time series anomaly detection helps to timely identify abnormal situations in the production process and ensure the quality and consistency of the product. For example, by analyzing the data collected by sensors on the production line, production parameters can be adjusted in time to improve production efficiency. In the monitoring of information technology systems, this technology can not only identify abnormal patterns in network traffic and assist security teams to respond quickly to network attacks, but also monitor key performance indicators to promptly detect system performance problems or potential failure points and ensure the stable operation of services. For example, by tracking indicators such as CPU usage, memory usage, and network latency of the server, problems can be discovered in time and measures can be taken to prevent service interruptions.

[0040] The importance of time series anomaly detection lies in its ability to help companies and institutions identify potential risks and problems in advance and take corresponding preventive measures. Compared with traditional periodic inspections and manual monitoring, this technology provides higher real-time and accuracy, can provide early warnings before problems occur, and significantly reduce losses caused by failures or abnormal situations. In addition, through in-depth analysis of abnormal data, companies can better understand the operating status of the system and possible problems, thereby optimizing management strategies and improving operational efficiency.

[0041] In the field of time series anomaly detection, a variety of models and techniques have been proposed to deal with the probability density distribution and anomaly detection problems of multidimensional data. The following are the anomaly detection methods in the existing related technologies:

[0042] A model combining deep autoencoders and Gaussian mixture models: This model can effectively model the probability density of multidimensional data.

[0043] Variational Autoencoder-based models: specially designed to detect anomalies in key performance indicator (KPI) time series with seasonal patterns, suitable for low-quality data.

[0044] Model integrating LSTM and variational autoencoder: Use the time series modeling ability of LSTM network and the data reconstruction ability of variational autoencoder to process complex time series data.

[0045] Model combining VAE and QRNNs: Through this combination, the model is able to capture long-term dependencies in time series and effectively handle time series data with nonlinear and non-Gaussian distribution characteristics.

[0046] Algorithms that apply the concept of spectral residuals to time series: The algorithm introduced by Microsoft enhances the performance of time series anomaly detection by combining convolutional neural networks.

[0047] Model based on normalized 2D images and ConvLSTM layers: This model captures the complex interactive information between time series by converting time series into images and using ConvLSTM layers to extract features.

[0048] Model combining CNN and bidirectional LSTM: This model is able to capture long-term trend changes, although it may face problems of high computational cost and poor scalability when dealing with high-dimensional data.

[0049] Model using random recurrent networks and normalized flows: This model models the correlation between random variables and makes anomaly judgments based on reconstruction probabilities.

[0050] Model with variational autoencoder as the core: This model considers the correlation between time series and each other at the same time, providing a comprehensive anomaly detection solution for multivariate time series.

[0051] Model that converts time series into 2D images: This method extracts local patterns and long-term correlation features, showing good anomaly detection results.

[0052] Generative Adversarial Network (GAN)-based model: By combining the autoencoder and GAN architecture and adopting an adversarial training approach, it can more accurately capture the intrinsic structure of the time series.

[0053] Model based on Transformer architecture: It can effectively use the self-attention mechanism to identify anomalies in high-dimensional and long sequence data.

[0054] Although these methods have achieved certain results in the field of anomaly detection, they still have limitations, mainly because they fail to fully consider the complexity of time series data and often rely only on the surface features of the data for prediction or reconstruction, but fail to deeply explore and utilize the deep characteristics of the data. This may lead to the model performing well in surface fitting, but has limitations in deep feature learning and understanding. In addition, time series data usually contains a lot of redundant information, which may interfere with the learning and generalization process of the model.

[0055] In order to solve the above problems, the present disclosure provides a multi-dimensional time series anomaly detection method, device, electronic device, storage medium and program product solution, which specifically include:

[0056] Determine a multidimensional time series, mask the multidimensional time series, and obtain a masked sequence; extract features from the multidimensional time series based on a time self-attention mechanism to obtain a first sequence feature; extract features from the multidimensional time series based on a feature self-attention mechanism to obtain a second sequence feature; reconstruct the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence; determine the error between the multidimensional time series and the reconstructed sequence as an anomaly score through a pre-trained anomaly detection model; judge the anomaly score based on a preset threshold, and determine that the multidimensional time series is abnormal in response to the anomaly score being greater than or equal to the preset threshold. The present disclosure divides the multidimensional time series into time series blocks (patches), randomly masks a part of them, and then uses the contrast learning structure of the Transformer model and the dual attention mechanism to extract features from the two perspectives of time series association and dimension association, trains the model to identify the feature representation of normal samples and reconstruct normal samples. In the detection stage, the model evaluates the consistency and reconstruction error of the sample to calculate the anomaly score, and judges whether the sample is abnormal according to the preset threshold, thereby achieving high-precision and stable multidimensional time series anomaly detection.

[0057] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0058] refer to Figure 1 , which is a schematic diagram of an application scenario of the multi-dimensional time series anomaly detection method provided by an exemplary embodiment of the present disclosure.

[0059] This application scenario includes a terminal device 101 and a server 102. The terminal device 101 and the server 102 may be connected via a wired or wireless communication network to achieve data interaction.

[0060] The terminal device 101 may be an electronic device with data transmission and multimedia input / output functions close to the user side, including but not limited to a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a personal digital assistant (PDA) or other electronic devices capable of realizing the above functions. The electronic device may include a processor and a display screen with a touch input function, the display screen is used to present a graphical user interface, the graphical user interface can display an application interface, and the processor is used to process application data, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0061] Server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0062] In some exemplary embodiments, the multi-dimensional time series anomaly detection method may be run on the terminal device 101 or the server 102 .

[0063] When the multi-dimensional time series anomaly detection method is run on the server 102 , the server 102 is used to provide a multi-dimensional time series anomaly detection service to a user of the terminal device 101 .

[0064] The server 102 receives the multidimensional time series transmitted from the terminal device 101, and the server 102 masks the multidimensional time series to obtain a masked sequence;

[0065] The server 102 extracts features from the multidimensional time series based on a temporal self-attention mechanism to obtain first sequence features;

[0066] The server 102 extracts features from the multidimensional time series based on a feature self-attention mechanism to obtain second sequence features;

[0067] The server 102 reconstructs the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence;

[0068] The server 102 determines the error between the multidimensional time series and the reconstructed sequence as an anomaly score through a pre-trained anomaly detection model;

[0069] The server 102 judges the anomaly score based on a preset threshold. In response to the anomaly score being greater than or equal to the preset threshold, the server 102 determines that an anomaly exists in the multidimensional time series, and transmits the detection result of determining that an anomaly exists in the multidimensional time series to the terminal device 101.

[0070] It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0071] refer to Figure 2 , a multidimensional time series anomaly detection method, the method comprising the following steps:

[0072] Step S210: determine a multidimensional time series, and mask the multidimensional time series to obtain a masked sequence.

[0073] In practical terms, multidimensional time series refers to:

[0074] A composite data structure consisting of multiple related time series data within a certain time range, each of which represents the change of a specific dimension or variable over time. These time series are aligned on the time axis and may be interdependent or correlated in different dimensions, together describing the dynamic behavior or state of a system.

[0075] In the above exemplary embodiment, a multidimensional time series is introduced. The following describes a method for obtaining a masked sequence:

[0076] In this exemplary embodiment, masking the multidimensional time series to obtain a masked sequence includes:

[0077] The multidimensional time series is segmented to obtain at least two original time series blocks; the quantity information of the original time series blocks is determined, and the length of the generated mask vector is determined based on the quantity information; the original time series blocks are masked based on the mask vector to obtain two complementary masked sequences corresponding to each original time series block. For example, in meteorology, it may include the changes of multiple meteorological elements such as temperature, humidity, and wind speed over time.

[0078] In a specific implementation, the multidimensional time series is segmented in a manner including but not limited to at least one of the following:

[0079] The multidimensional time series is segmented by a sliding window, a segmentation method based on dynamic programming, or a segmentation method based on principal component analysis (PCA) and visibility graph theory.

[0080] In a specific implementation, the multidimensional time series is segmented to obtain at least two original time series blocks; the quantity information of the original time series blocks is determined, and the length of the generated mask vector is determined based on the quantity information; the original time series blocks are masked based on the mask vector to obtain two complementary masked sequences corresponding to each of the original time series blocks:

[0081] Each input multidimensional time series sequence is divided into multiple time series blocks (patches), and a mask vector with a mask rate of 50% is generated. The length of the vector is the number of patches of the input sequence. Through the Mask vector, two complementary masked sequences can be generated for each sequence; that is, the length of the input multidimensional time series sequence is determined, denoted as L; a vector of length L is created, and all elements are initially set to 1 (indicating unmasked); randomly select Positions, set the element values ​​of these positions to 0 (indicating a mask); ensure that when selecting the mask position, the selection is random to ensure that each generated Mask vector is different, which can increase the generalization ability of the model; since two complementary masked sequences are required, you can create another vector that is opposite to the original Mask vector, that is, the position that is 1 in the original vector is 0 in the new vector, and the position that is 0 in the original vector is 1 in the new vector. The formula is as follows:

[0082]

[0083] Where w∈R M×N×P is the input sequence, Mask∈R N is the mask vector, is the masked sequence; where M refers to the number of multiple independent sequences processed simultaneously, N refers to the time series length of a single sequence in the time series, P refers to the length or size of each patch when splitting the time series, and R is a real number set.

[0084] Step S220: extract features of the multidimensional time series based on a temporal self-attention mechanism to obtain first sequence features.

[0085] In specific implementation, the temporal self-attention mechanism refers to:

[0086] The temporal self-attention mechanism focuses on capturing the dependencies between different time points within a single time series. It determines the degree of influence of each time point in the sequence on other time points by mapping each patch of the time series into a query, key, and value vector, and then calculating the attention score between these vectors. This process allows the model to dynamically adjust the degree of attention to each time point, thereby learning the feature representation and temporal association within the sequence.

[0087] In the above exemplary embodiment, the temporal self-attention mechanism is introduced. The following is an introduction to the method of obtaining the first sequence feature:

[0088] In this exemplary embodiment, the masked sequence is subjected to feature extraction based on a temporal self-attention mechanism to obtain a first sequence feature, including:

[0089] An embedding operation is performed on the original time series block to obtain a sequence embedding vector; a unidimensional time series of the multidimensional time series is determined, and a first query matrix is ​​obtained based on the unidimensional time series and a first learning parameter; a first key matrix is ​​obtained based on the unidimensional time series and a second learning parameter; a first value matrix is ​​obtained based on the unidimensional time series and a third learning parameter; the sequence embedding vector is normalized based on the first query matrix and the first key matrix to obtain a first attention weight; a first weighting matrix is ​​obtained based on the first attention weight and the first value matrix; the first weighting matrix is ​​spliced ​​to obtain a first self-attention output; the first sequence feature is obtained by splicing based on the unidimensional time series and the first self-attention output and performing linear transformation.

[0090] In a specific implementation, the embedding operation is performed on the original time series block to obtain a sequence embedding vector:

[0091] Each original time series block of length P is converted into a vector of fixed dimension through embedding operation, namely the sequence embedding vector d model .

[0092] In a specific implementation, determining the single-dimensional time series of the multi-dimensional time series refers to:

[0093] In a multidimensional time series, each dimension represents a univariate time series. You can determine a unidimensional time series by selecting a specific dimension. For example, if you have a multidimensional time series data containing temperature, humidity, and pressure, you can select the temperature dimension and analyze it as a unidimensional time series. Use methods such as statistics, spectral analysis, or wavelet transform to extract unidimensional features from a multidimensional time series. For example, you can extract the main frequency components through Fourier transform, or use wavelet transform to extract key wavelet coefficients as features of a unidimensional time series.

[0094] In a specific implementation, based on the one-dimensional time series and the first learning parameter, a first query matrix is ​​obtained; based on the one-dimensional time series and the second learning parameter, a first key matrix is ​​obtained; based on the one-dimensional time series and the third learning parameter, a first value matrix is ​​obtained, in the following manner:

[0095] Select data from a specific single-dimensional time series from a multidimensional time series and are learnable parameters, which are weight matrices used to transform a single-dimensional time series Converted to query, key and value matrix.

[0096]

[0097] In the formula, are the query, key, and value matrices of the multi-head self-attention mechanism, respectively. They are is a learnable parameter, and h is the number of heads in the multi-head attention mechanism.

[0098] In a specific implementation, the sequence embedding vector is normalized based on the first query matrix and the first key matrix to obtain a first attention weight; based on the first attention weight and the first value matrix, a first weighting matrix is ​​obtained:

[0099] is the query matrix and key matrix Transpose The dot product of First attention weight It can be expressed by the following formula:

[0100]

[0101] The first weighting matrix It can be expressed by the following formula:

[0102]

[0103] Where Softmax(·) is used to standardize the self-attention weights.

[0104] In a specific implementation, the first weighted matrix is ​​spliced ​​to obtain a first self-attention output; based on the one-dimensional time series and the first self-attention output, the first sequence feature Z is obtained by splicing and linear transformation. N can be expressed by the following formula:

[0105]

[0106] In the formula, are learnable parameters.

[0107] Step S230: extract features from the multidimensional time series based on a feature self-attention mechanism to obtain second sequence features.

[0108] In specific implementation, the feature self-attention mechanism refers to:

[0109] Focuses on processing the internal correlation of each dimension (or feature) in a multidimensional time series. It also determines the interdependence between different features by mapping each feature (or dimension) into query, key, and value vectors, and then calculating the attention scores between these vectors. This mechanism helps the model understand the relationship between different features and integrate these features to form a richer feature representation.

[0110] In the above exemplary embodiment, a feature self-attention mechanism is introduced. Next, a method for obtaining the second sequence feature is introduced:

[0111] In this exemplary embodiment, the multidimensional time series is subjected to feature extraction based on a feature self-attention mechanism to obtain a second sequence feature, including:

[0112] Based on the multidimensional time series and the first learning parameters, a second query matrix is ​​obtained; based on the multidimensional time series and the second learning parameters, a second key matrix is ​​obtained; based on the multidimensional time series and the third learning parameters, a second value matrix is ​​obtained; based on the second query matrix and the second key matrix, the sequence embedding vector is normalized to obtain a second attention weight; based on the second attention weight and the second value matrix, a second weighting matrix is ​​obtained; the second weighting matrix is ​​spliced ​​to obtain a second self-attention output; based on the multidimensional time series and the second self-attention output, splicing is performed and the second sequence feature is obtained through linear transformation.

[0113] In a specific implementation, a second query matrix is ​​obtained based on the multidimensional time series and the first learning parameter; a second key matrix is ​​obtained based on the multidimensional time series and the second learning parameter; and a second value matrix is ​​obtained based on the multidimensional time series and the third learning parameter:

[0114] and are learnable parameters, which are weight matrices used to transform the multidimensional time series data X i Converted to query, key and value matrix.

[0115]

[0116] In the formula, They are respectively the query, key, and value of the multi-head self-attention mechanism. They are is a learnable parameter, and h is the number of heads in the multi-head attention mechanism.

[0117] In a specific implementation, the sequence embedding vector is normalized based on the second query matrix and the second key matrix to obtain a second attention weight; based on the second attention weight and the second value matrix, a second weighting matrix is ​​obtained:

[0118] is the query matrix and key matrix Transpose The dot product of First attention weight It can be expressed by the following formula:

[0119]

[0120] The first weighting matrix It can be expressed by the following formula:

[0121]

[0122] Where Softmax(·) is used to standardize the self-attention weights.

[0123] In a specific implementation, the second weighted matrix is ​​spliced ​​to obtain a second self-attention output; based on the multidimensional time series and the second self-attention output, the second sequence feature Z is obtained by splicing and linear transformation. M It can be expressed by the following formula:

[0124]

[0125] In the formula, are learnable parameters.

[0126] Step S240: reconstruct the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence.

[0127] In this exemplary embodiment, the first sequence feature and the second sequence feature are concatenated based on the masked sequence to obtain a reconstructed sequence, including:

[0128] The first sequence feature and the second sequence feature are feature fused to obtain a fused time series feature; the fused time series feature is spliced ​​with the masked sequence to obtain a fused feature sequence; the original order of the feature fusion sequence is restored to obtain an aligned fused feature sequence; the spatiotemporal position embedding information of the multidimensional time series is determined, the spatiotemporal position embedding information is added to the aligned fused feature sequence and decoded to obtain the reconstructed sequence.

[0129] In a specific implementation, the first sequence feature and the second sequence feature are fused to obtain a fused time series feature:

[0130] The first sequence feature X is transformed into N The first self-attention output Z in N With the second sequence feature X M The second self-attention output Z in N The fusion is performed through the encoder to produce a richer information representation, where the fused time series feature Z can be expressed by the following formula:

[0131] Z=Fuse(Z N ,Z M ).

[0132] In a specific implementation, the fused time series features are spliced ​​with the masked sequence to obtain a fused feature sequence; the original order of the feature fusion sequence is restored to obtain an aligned fused feature sequence; the spatiotemporal position embedding information of the multidimensional time series is determined, the spatiotemporal position embedding information is added to the aligned fused feature sequence and decoded to obtain the reconstructed sequence:

[0133] The masked patches of the fused time series feature Z and the masked sequence are concatenated, and the unshuffle operation is performed to restore the order. After adding the spatiotemporal position embedding information, they are input into the decoder together. Each masked patch is a learnable vector, which indicates that this is a patch to be reconstructed, where the reconstructed sequence It can be expressed by the following formula:

[0134]

[0135] In the formula, is the reconstructed output of X.

[0136] Step S250: Determine the error between the multidimensional time series and the reconstructed sequence using a pre-trained anomaly detection model as an anomaly score.

[0137] In this exemplary embodiment, the anomaly detection model is trained by the following method:

[0138] Construct a sample set including several samples; wherein the samples include: sample data and label data; the sample data includes a training multidimensional time series; the label data includes a training anomaly score corresponding to the multidimensional time series; the training multidimensional time series is masked to obtain a training masked sequence; based on a temporal self-attention mechanism, feature extraction is performed on the training multidimensional time series to obtain a training first sequence feature; based on a feature self-attention mechanism, feature extraction is performed on the training multidimensional time series to obtain a training second sequence feature; based on the training first sequence feature and the training second sequence feature, the training masked sequence is reconstructed to obtain a training reconstructed sequence; by waiting for The method comprises the following steps: providing a training anomaly detection model, determining an error between the training multidimensional time series and the training reconstruction sequence as a prediction score; determining a difference between the prediction score and the training anomaly score; constructing a consistency loss function based on the training first sequence features and the training second sequence; constructing a reconstruction loss function based on the training multidimensional time series and the training reconstruction sequence; combining the consistency loss function with the reconstruction loss function to obtain a final loss function; and updating parameters of the anomaly detection model to be trained based on the difference through the final loss function until the difference between the prediction score and the training anomaly score is minimized to obtain the pre-trained anomaly detection model.

[0139] In a specific implementation, a consistency loss function is constructed based on the first training sequence features and the second training sequence:

[0140] The consistency loss uses KL divergence to measure the first sequence feature (Z N ′) and the second sequence feature for training (Z M ′). KL divergence is a method to measure the difference between two probability distributions, and is used here to measure the difference between feature representations obtained from different perspectives. The definition of consistency loss is as follows:

[0141]

[0142] In the formula, KL(·||·) is used to calculate the divergence, Z M′ , Z N′ They are respectively the inter-sequence feature representation for training and the intra-sequence feature representation for training, ConLoss∈R L×1 The difference between the two viewpoints for each point in the input time series is used for training.

[0143] In a specific implementation, a reconstruction loss function is constructed based on the training multidimensional time series and the training reconstruction sequence:

[0144] The reconstruction loss uses the L2 norm to calculate the reconstructed output and the masked sequence (w m The L2 norm is a method to measure the size of a vector, which is used here to measure the difference between the sequence reconstructed by the model and the actual sequence. The definition of reconstruction loss is as follows:

[0145]

[0146] is the reconstructed output of X′, and ‖·‖2 indicates the use of the L2 norm.

[0147] In specific implementation, the consistency loss function is combined with the reconstruction loss function to obtain the final loss function:

[0148] The final loss function is the weighted sum of the reconstruction loss and the consistency loss, where λ is a hyperparameter used to balance the contribution between the two losses. The final loss function is defined as follows:

[0149]

[0150] Step S260: judging the anomaly score based on a preset threshold, and in response to the anomaly score being greater than or equal to the preset threshold, determining that an anomaly exists in the multidimensional time series.

[0151] In a specific implementation, judging the anomaly score based on a preset threshold, and in response to the anomaly score being greater than or equal to the preset threshold, determining that the multidimensional time series has an anomaly means:

[0152] Because normal samples account for the vast majority of the model's training data, the model can reconstruct normal samples very well, but the reconstruction effect of abnormal samples is not ideal. Therefore, when normal samples are input into the model, a smaller anomaly score will be obtained, and when abnormal samples are input into the model, a larger anomaly score will be obtained. After calculating the anomaly scores of all test samples, a threshold can be set. When the anomaly score S of the test sample is greater than or equal to When the abnormal score S of the test sample is less than It is judged as normal when the

[0153]

[0154] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0155] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0156] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a multi-dimensional time series anomaly detection device.

[0157] refer to Figure 3 , the multidimensional time series anomaly detection device comprises:

[0158] The mask sequence determination module 310 is configured to determine a multidimensional time series, mask the multidimensional time series, and obtain a masked sequence;

[0159] A first feature determination module 320 is configured to extract features from the multidimensional time series based on a temporal self-attention mechanism to obtain a first sequence feature;

[0160] The second feature determination module 330 is configured to extract features from the multidimensional time series based on a feature self-attention mechanism to obtain a second sequence feature;

[0161] A reconstructed sequence determination module 340 is configured to reconstruct the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence;

[0162] an anomaly score determination module 350, configured to determine the error between the multidimensional time series and the reconstructed sequence as an anomaly score through a pre-trained anomaly detection model;

[0163] The detection result determination module 360 ​​is configured to judge the abnormality score based on a preset threshold, and determine that an abnormality exists in the multidimensional time series in response to the abnormality score being greater than or equal to the preset threshold.

[0164] In this exemplary embodiment, the mask sequence determination module 310 is specifically configured as follows:

[0165] Determine a multidimensional time series, segment the multidimensional time series to obtain at least two original time series blocks; determine the quantity information of the original time series blocks, and determine the length of the generated mask vector based on the quantity information; mask the original time series blocks based on the mask vector to obtain two complementary masked sequences corresponding to each of the original time series blocks.

[0166] In this exemplary embodiment, the first feature determination module 320 is specifically configured as follows:

[0167] An embedding operation is performed on the original time series block to obtain a sequence embedding vector; a unidimensional time series of the multidimensional time series is determined, and a first query matrix is ​​obtained based on the unidimensional time series and a first learning parameter; a first key matrix is ​​obtained based on the unidimensional time series and a second learning parameter; a first value matrix is ​​obtained based on the unidimensional time series and a third learning parameter; the sequence embedding vector is normalized based on the first query matrix and the first key matrix to obtain a first attention weight; a first weighting matrix is ​​obtained based on the first attention weight and the first value matrix; the first weighting matrix is ​​spliced ​​to obtain a first self-attention output; the first sequence feature is obtained by splicing based on the unidimensional time series and the first self-attention output and performing linear transformation.

[0168] In this exemplary embodiment, the second feature determination module 330 is specifically configured as follows:

[0169] Based on the multidimensional time series and the first learning parameters, a second query matrix is ​​obtained; based on the multidimensional time series and the second learning parameters, a second key matrix is ​​obtained; based on the multidimensional time series and the third learning parameters, a second value matrix is ​​obtained; based on the second query matrix and the second key matrix, the sequence embedding vector is normalized to obtain a second attention weight; based on the second attention weight and the second value matrix, a second weighting matrix is ​​obtained; the second weighting matrix is ​​spliced ​​to obtain a second self-attention output; based on the multidimensional time series and the second self-attention output, splicing is performed and the second sequence feature is obtained through linear transformation.

[0170] In this exemplary embodiment, the reconstruction sequence determination module 340 is specifically configured as follows:

[0171] The first sequence feature and the second sequence feature are feature fused to obtain a fused time series feature; the fused time series feature is spliced ​​with the masked sequence to obtain a fused feature sequence; the original order of the feature fusion sequence is restored to obtain an aligned fused feature sequence; the spatiotemporal position embedding information of the multidimensional time series is determined, the spatiotemporal position embedding information is added to the aligned fused feature sequence and decoded to obtain the reconstructed sequence.

[0172] In this exemplary embodiment, the anomaly score determination module 350 is specifically configured to:

[0173] The anomaly detection model is trained by the following method: constructing a sample set including several samples; wherein the samples include: sample data and label data; the sample data includes a multidimensional time series for training; the label data includes a training anomaly score corresponding to the multidimensional time series; the training multidimensional time series is masked to obtain a masked training sequence; based on a temporal self-attention mechanism, feature extraction is performed on the training multidimensional time series to obtain a first training sequence feature; based on a feature self-attention mechanism, feature extraction is performed on the training multidimensional time series to obtain a second training sequence feature; based on the first training sequence feature and the second training sequence feature, the masked training sequence is reconstructed to obtain a reconstructed training sequence; through the anomaly detection model to be trained, it is determined The invention relates to a method for detecting anomalies in a pre-trained anomaly detection model. The method comprises the steps of: determining an error between the training multidimensional time series and the training reconstructed sequence as a prediction score; determining a difference between the prediction score and the training anomaly score; constructing a consistency loss function based on the training first sequence features and the training second sequence; constructing a reconstruction loss function based on the training multidimensional time series and the training reconstructed sequence; combining the consistency loss function with the reconstruction loss function to obtain a final loss function; based on the difference, updating the parameters of the anomaly detection model to be trained through the final loss function until the difference between the prediction score and the training anomaly score is minimized, thereby obtaining the pre-trained anomaly detection model; determining an error between the multidimensional time series and the reconstructed sequence as an anomaly score through the pre-trained anomaly detection model.

[0174] In this exemplary embodiment, the detection result determination module 360 ​​is specifically configured as follows:

[0175] The abnormality score is judged based on a preset threshold, and in response to the abnormality score being greater than or equal to the preset threshold, it is determined that the multidimensional time series has an abnormality.

[0176] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0177] The device of the above embodiment is used to implement the corresponding multi-dimensional time series anomaly detection method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0178] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multidimensional time series anomaly detection method described in any of the above embodiments is implemented.

[0179] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0180] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0181] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0182] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0183] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0184] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0185] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0186] The electronic device of the above embodiment is used to implement the corresponding multi-dimensional time series anomaly detection method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0187] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the multidimensional time series anomaly detection method as described in any of the above embodiments.

[0188] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0189] The above-mentioned non-transitory computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0190] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the multidimensional time series anomaly detection method described in any embodiment in the above exemplary method part, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0191] Based on the same inventive concept, corresponding to the multidimensional time series anomaly detection method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the multidimensional time series anomaly detection method. Corresponding to the execution subject corresponding to each step in each embodiment of the multidimensional time series anomaly detection method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0192] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the multidimensional time series anomaly detection method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0193] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." In addition, in some embodiments, the present disclosure may also be implemented in the form of a computer program product in one or more computer-readable media, which contains computer-readable program code.

[0194] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive examples) of computer-readable storage media may include, for example: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0195] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0196] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0197] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0198] It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, and these computer program instructions are executed by a computer or other programmable data processing device to produce a device that implements the functions / operations specified in the boxes in the flowchart and / or block diagram.

[0199] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable medium produce a product that includes an instruction device that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0200] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby enabling the instructions executed on the computer or other programmable device to provide a process for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0201] In addition, although the operations of the disclosed method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0202] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0203] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0204] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0205] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0206] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0207] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

[0208] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims. The scope of the attached claims conforms to the broadest interpretation, thereby including all such modifications and equivalent structures and functions.

Claims

1. A multidimensional time series anomaly detection method, characterized in that: include: Determine a multidimensional time series, and mask the multidimensional time series to obtain a masked sequence; Extracting features from the multidimensional time series based on a temporal self-attention mechanism to obtain first sequence features; Extracting features from the multidimensional time series based on a feature self-attention mechanism to obtain second sequence features; Reconstructing the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence; Determine, by a pre-trained anomaly detection model, an error between the multidimensional time series and the reconstructed sequence as an anomaly score; The abnormality score is judged based on a preset threshold, and in response to the abnormality score being greater than or equal to the preset threshold, it is determined that the multidimensional time series has an abnormality.

2. The method according to claim 1, characterized in that The step of masking the multidimensional time series to obtain a masked sequence includes: Segmenting the multidimensional time series to obtain at least two original time series blocks; Determine quantity information of the original timing blocks, and determine the length of the generated mask vector based on the quantity information; The original timing blocks are masked based on the mask vector to obtain two complementary masked sequences corresponding to each of the original timing blocks.

3. The method according to claim 2, characterized in that The extracting features of the multidimensional time series based on the time self-attention mechanism to obtain the first sequence features includes: Performing an embedding operation on the original time series block to obtain a sequence embedding vector; Determine a single-dimensional time series of the multidimensional time series, and obtain a first query matrix based on the single-dimensional time series and a first learning parameter; Based on the one-dimensional time series and the second learning parameter, a first key matrix is ​​obtained; Based on the one-dimensional time series and the third learning parameter, a first value matrix is ​​obtained; Normalizing the sequence embedding vector based on the first query matrix and the first key matrix to obtain a first attention weight; Obtaining a first weighting matrix based on the first attention weight and the first value matrix; Concatenating the first weighted matrices to obtain a first self-attention output; The first sequence feature is obtained by concatenating the one-dimensional time series and the first self-attention output and performing linear transformation.

4. The method according to claim 3, characterized in that The step of extracting features from the multidimensional time series based on the feature self-attention mechanism to obtain a second sequence feature includes: Based on the multidimensional time series and the first learning parameter, obtaining a second query matrix; Based on the multidimensional time series and the second learning parameter, obtaining a second key matrix; Based on the multidimensional time series and the third learning parameter, obtaining a second value matrix; Normalizing the sequence embedding vector based on the second query matrix and the second key matrix to obtain a second attention weight; Based on the second attention weight and the second value matrix, obtaining a second weighting matrix; Concatenating the second weighted matrix to obtain a second self-attention output; The second sequence feature is obtained by concatenating the multidimensional time series and the second self-attention output and performing linear transformation.

5. The method according to claim 1, characterized in that The step of splicing the first sequence feature and the second sequence feature based on the masked sequence to obtain a reconstructed sequence includes: Performing feature fusion on the first sequence feature and the second sequence feature to obtain a fused time series feature; splicing the fused time series features and the masked sequence to obtain a fused feature sequence; Restoring the original order of the feature fusion sequence to obtain an aligned fusion feature sequence; Determine the spatiotemporal position embedding information of the multidimensional time series, add the spatiotemporal position embedding information into the aligned fusion feature sequence and decode it to obtain the reconstructed sequence.

6. The method according to claim 1, characterized in that The anomaly detection model further comprises training the anomaly detection model by: Constructing a sample set including a plurality of samples; wherein the samples include: sample data and label data; the sample data includes a multidimensional time series for training; the label data includes an anomaly score for training corresponding to the multidimensional time series; Masking the training multidimensional time series to obtain a masked training sequence; Extracting features from the multidimensional time series for training based on a temporal self-attention mechanism to obtain first sequence features for training; Extracting features from the training multidimensional time series based on a feature self-attention mechanism to obtain a second sequence feature for training; Reconstructing the masked training sequence based on the first training sequence feature and the second training sequence feature to obtain a reconstructed training sequence; Determining, by means of the anomaly detection model to be trained, an error between the training multidimensional time series and the training reconstructed sequence as a prediction score; determining a difference between the predicted score and the training anomaly score; Constructing a consistency loss function based on the first training sequence features and the second training sequence; Constructing a reconstruction loss function based on the training multidimensional time series and the training reconstruction sequence; Combining the consistency loss function with the reconstruction loss function to obtain a final loss function; Based on the difference, the parameters of the anomaly detection model to be trained are updated by the final loss function until the difference between the predicted score and the training anomaly score is minimized, thereby obtaining the pre-trained anomaly detection model.

7. A multidimensional time series anomaly detection device, characterized in that: include: a mask sequence determination module, configured to determine a multidimensional time series, mask the multidimensional time series, and obtain a masked sequence; A first feature determination module is configured to extract features from the multidimensional time series based on a temporal self-attention mechanism to obtain a first sequence feature; A second feature determination module is configured to extract features from the multidimensional time series based on a feature self-attention mechanism to obtain a second sequence feature; a reconstructed sequence determining module, configured to reconstruct the masked sequence based on the first sequence feature and the second sequence feature to obtain a reconstructed sequence; an anomaly score determination module, configured to determine an error between the multidimensional time series and the reconstructed sequence as an anomaly score through a pre-trained anomaly detection model; The detection result determination module is configured to judge the abnormality score based on a preset threshold, and in response to the abnormality score being greater than or equal to the preset threshold, determine that the multidimensional time series has an abnormality.

8. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer program instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 6.

Images