A supervised earthquake monitoring method and device for time series data
The supervised earthquake monitoring method addresses the limitations of existing anomaly detection by using a continuous-time self-attention encoder and decoder for irregular time series, enabling effective advance detection and adaptation to changing data.
Patent Information
- Application Number
- CN202411321683.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2044-09-23
AI Technical Summary
Existing anomaly detection methods are suboptimal for irregular time series and lack advance detection capabilities, failing to adapt to long-term dynamically changing data and unseen anomalies.
A supervised earthquake monitoring method using a continuous-time self-attention encoder and self-attention decoder for encoding and decoding information between windows, combined with multi-task learning and knowledge distillation, to support both traditional and advance anomaly detection scenarios.
The method effectively encodes and decodes information across different windows, actively learning to overcome unseen anomalies and data drift, providing advance anomaly detection for irregular time series.
Smart Images
Figure CN118836977B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of anomaly detection, and in particular relates to a supervised earthquake monitoring method and device for time series data. Background Art
[0002] With the rapid development of different sensors, large-scale time series data are collected in many different applications, such as earthquake monitoring, finance, and network security. Effectively discovering abnormal patterns is crucial to ensuring safety and avoiding economic losses.
[0003] There has been a lot of work on anomaly detection methods combined with regular time series in recent years. However, time series data is often irregular, and many existing anomaly detection methods designed for regular time series are often suboptimal when applied to irregular time series. At the same time, existing anomaly detection scenarios are all detections after data arrives, which has a lag and cannot provide foresight. Combining irregular time series with advance anomaly detection scenarios can provide advance detection and perform pre-disaster recovery, greatly expanding the application value of anomaly detection. Therefore, designing an advance anomaly detection method for time series data has become an urgent need in academia and industry.
[0004] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:
[0005] First, most existing research is limited to the field of regular time series and traditional anomaly detection without advance time, and has not fully explored the application potential of irregular time series and advance anomaly detection. In addition, existing methods ignore the anomalies that have not been seen when the model is online and the concept drift of data, and cannot adapt to long-term dynamically changing data, and the effect is not good. Summary of the invention
[0006] In view of the shortcomings of the prior art, the purpose of the embodiments of the present application is to provide a supervised earthquake monitoring method and device for time series data, which explores the application potential of combining irregular data with early anomaly detection scenarios and overcomes the problem of poor model caused by long-term dynamically changing data to a certain extent.
[0007] According to a first aspect of an embodiment of the present application, a supervised earthquake monitoring method for time series data is provided, comprising:
[0008] Acquire earthquake monitoring time series data, wherein the earthquake monitoring time series data includes earthquake monitoring data at several time points and corresponding real label data;
[0009] According to the earthquake monitoring time series data, the current window and the prediction window are divided to construct a training set, wherein the training set includes a training window set and a training window true label set;
[0010] Based on the training set, the hidden layer representations of the current window and the prediction window are obtained by encoding through a continuous time self-attention encoder, and the current window label is obtained by decoding through a shared linear layer;
[0011] Merge the all-0 window with the second half of the current window to form a merged window with the same length as the current window, input the merged window into the self-attention decoder, use the hidden layer representation of the current window to decode and generate the predicted hidden layer representation of the prediction window, and obtain the predicted window label through the shared linear layer;
[0012] Based on the real label set of the training window, the label of the current window, and the label of the predicted window, the label loss is calculated, the distillation loss is calculated based on the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window, and the continuous time self-attention encoder, the self-attention decoder, and the shared linear layer are trained according to the label loss and the distillation loss;
[0013] Real-time earthquake monitoring data is obtained, and the trained continuous-time self-attention encoder, self-attention decoder, and shared linear layer are used to predict earthquake conditions within a predetermined time in the future.
[0014] Furthermore, according to the earthquake monitoring time series data, the current window and the prediction window are divided to construct a training set, including:
[0015] Based on the current window size , prediction window size With moving step length Split the earthquake monitoring time series data to obtain the current window set With the prediction window set ,in , Represents time The initial window size is The current window, Represents time The initial window size is The prediction window of
[0016] According to the current sliding window size , prediction window size With moving step length Split the real label data corresponding to the earthquake monitoring time series data to obtain the real label set of the current window and the true label set of the prediction window ;
[0017] Set the current window With the prediction window set , the current window real label set and the true label set of the prediction window Merge respectively to construct a training set, the training set includes a training window set and the training window true label set .
[0018] Furthermore, the continuous-time self-attention encoder includes several encoding layers, each encoding layer includes a first interpolation module, a first definite integral linear layer, a second definite integral linear layer, a continuous-time multi-head attention layer, a first residual connection and normalization layer, a first feedforward fully connected network layer and a second residual connection and normalization layer. For the input sequence data, the continuous-time self-attention encoder performs the following processing:
[0019] The interpolation module fits the sequence data through natural cubic curve interpolation and resamples to obtain a first query vector;
[0020] The first definite integral linear layer and the second definite integral linear layer calculate the first key vector and the first value vector respectively;
[0021] The continuous-time multi-head attention layer captures the time evolution relationship between the observations according to the first query vector, the first key vector and the first value vector to obtain a continuous-time hidden layer representation;
[0022] The continuous-time hidden layer representation obtains the hidden layer representation of the sequence data through a continuous first residual connection and normalization layer, a first feedforward fully connected network layer and a second residual connection and normalization layer.
[0023] Furthermore, the shared linear layer is used to decode the hidden layer representation of the current window to obtain the current window label ,in , and Represent average pooling and maximum pooling respectively, is the hidden layer representation of the current window, Indicates a linear transformation of the current window aggregate embedding, The offset to adjust the result by.
[0024] Furthermore, the self-attention decoder includes several decoding layers, each decoding layer includes a second interpolation module, a self-attention layer, a third residual connection and normalization layer, a cross attention layer, a fourth residual connection and normalization layer, a second feedforward fully connected network layer, a fifth residual connection and normalization layer, and for the input sequence data, the self-attention decoder performs the following processing:
[0025] For non-0 segment data in the sequence data, the second interpolation module performs fitting and resampling through natural cubic curve interpolation and splicing with the 0 segment data in the sequence, and obtains a second query vector, a second key vector and a second value vector through embedding and linear transformation;
[0026] A preliminary hidden layer representation is obtained through a third residual connection and a normalization layer as a third query vector;
[0027] Based on the hidden layer representation of the current window, a third key vector and a third value vector are obtained, which are decoded through a cross attention layer, and passed through a fourth residual connection and normalization layer, a second feedforward fully connected network layer, and a fifth residual connection and normalization layer to obtain a predicted hidden layer representation corresponding to the sequence data.
[0028] Further, based on the real label set of the training window, the label of the current window, and the label of the predicted window, a label loss is calculated, a distillation loss is calculated based on the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window, and the continuous time self-attention encoder, the self-attention decoder, and the shared linear layer are trained according to the label loss and the distillation loss, including:
[0029] Calculate the first label loss , as the first loss; among them represents the cross entropy loss function, is the real label of the current window, The current window label;
[0030] Calculate the second label loss ,in is the true label of the prediction window, is the prediction window label;
[0031] Calculating distillation losses ,in To predict the window size, represents the KL divergence, refers to the normalized exponential function, They represent the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window respectively;
[0032] A second loss is obtained by weighted summing the second label loss and the distillation loss;
[0033] The continuous-time self-attention encoder and the shared linear layer are back-propagated based on the first loss, and the self-attention decoder and the shared linear layer are back-propagated based on the second loss, thereby achieving training.
[0034] Furthermore, during the real-time prediction process, a test set is constructed and the shared linear layer is fine-tuned on the test set. The construction process of the test set is specifically as follows:
[0035] Calculate the margin based on the predicted probability ,in represents the probability of being predicted as an anomaly, It represents the probability of being predicted as normal, and the data with the smallest marginal value is selected and saved into the test set, where the size of the test set is constant, and the data therein is replaced according to the first-in-first-out strategy.
[0036] According to a second aspect of an embodiment of the present application, a supervised earthquake monitoring device for time series data is provided, comprising:
[0037] An acquisition module is used to acquire earthquake monitoring time series data, wherein the earthquake monitoring time series data includes earthquake monitoring data at several time points and corresponding real label data;
[0038] A training set construction module, used to divide the current window and the prediction window according to the earthquake monitoring time series data, so as to construct a training set, wherein the training set includes a training window set and a training window true label set;
[0039] A first hidden layer representation generation module is used to obtain hidden layer representations of the current window and the prediction window based on the training set by encoding through a continuous time self-attention encoder, and obtain a current window label by decoding through a shared linear layer;
[0040] The second hidden layer representation generation module is used to merge the all-0 window with the second half of the current window to form a merged window with the same length as the current window, input the merged window into the self-attention decoder, use the hidden layer representation of the current window to decode and generate the predicted hidden layer representation of the prediction window, and obtain the predicted window label through the shared linear layer;
[0041] A training module, used to calculate the label loss based on the real label set of the training window, the current window label, and the predicted window label, calculate the distillation loss based on the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window, and train the continuous time self-attention encoder, self-attention decoder and shared linear layer according to the label loss and distillation loss;
[0042] The prediction module is used to obtain real-time earthquake monitoring data and use the trained continuous-time self-attention encoder, self-attention decoder and shared linear layer to predict earthquake conditions within a predetermined time in the future.
[0043] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including:
[0044] one or more processors;
[0045] A memory for storing one or more programs;
[0046] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect.
[0047] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.
[0048] The technical solution provided by the embodiments of the present application may have the following beneficial effects:
[0049] It can be seen from the above implementation that the present application constructs an advance anomaly detection method and device for time series data that can fully encode and decode information between different windows. The present application uses a continuous-time self-attention encoder and a self-attention decoder to build a framework for encoding and decoding information from different windows. Through multi-task learning and knowledge distillation, it supports both traditional anomaly detection scenarios and advance anomaly detection scenarios. It uses relatively simpler traditional anomaly detection scenarios to improve the effect of advance anomaly detection scenarios, and after going online, it actively learns through incremental fine-tuning, which overcomes the problems of unseen anomalies and data offset to a certain extent.
[0050] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0052] Figure 1 It is a flowchart of a supervised earthquake monitoring method for time series data according to an exemplary embodiment.
[0053] Figure 2 is a schematic diagram of a continuous-time self-attention encoder, a self-attention decoder, and a shared linear layer according to an exemplary embodiment.
[0054] Figure 3 It is a block diagram of a supervised earthquake monitoring device for time series data according to an exemplary embodiment.
[0055] Figure 4 The diagram is a schematic diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0056] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.
[0057] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0058] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0059] Figure 1 is a flowchart of a supervised earthquake monitoring method for time series data according to an exemplary embodiment. Figure 1 As shown, the method may include the following steps:
[0060] S11: Acquire earthquake monitoring time series data, where the earthquake monitoring time series data includes earthquake monitoring data at several time points and corresponding real label data;
[0061] S12: dividing the current window and the prediction window according to the earthquake monitoring time series data, thereby constructing a training set, wherein the training set includes a training window set and a training window true label set;
[0062] S13: Based on the training set, obtain the hidden layer representation of the current window and the prediction window by encoding through the continuous time self-attention encoder, and obtain the current window label by decoding through the shared linear layer;
[0063] S14: merge the all-0 window with the second half of the current window to form a merged window with the same length as the current window, input the merged window into the self-attention decoder, use the hidden layer representation of the current window to decode and generate the predicted hidden layer representation of the prediction window, and obtain the predicted window label through the shared linear layer;
[0064] S15: Calculate label loss based on the real label set of the training window, the label of the current window, and the label of the prediction window, calculate distillation loss based on the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window, and train the continuous time self-attention encoder, self-attention decoder, and shared linear layer according to the label loss and distillation loss;
[0065] S16: Obtain real-time earthquake monitoring data and use the trained continuous-time self-attention encoder, self-attention decoder, and shared linear layer to predict earthquake conditions within a predetermined time in the future.
[0066] From the above implementation, it can be seen that the present application constructs an advance anomaly detection method and device for time series data that can fully encode and decode information between different windows. The present application uses a continuous-time self-attention model and a self-attention decoding module to build a framework for encoding and decoding information from different windows. Through multi-task learning and knowledge distillation, it supports both traditional anomaly detection scenarios and advance anomaly detection scenarios. It uses relatively simple traditional anomaly detection scenarios to improve the effect of advance anomaly detection scenarios, and after going online, it actively learns through incremental fine-tuning, which overcomes the problems of unseen anomalies and data offset to a certain extent.
[0067] In a specific implementation of S11, earthquake monitoring time series data is obtained, where the earthquake monitoring time series data includes earthquake monitoring data at several time points and corresponding real label data;
[0068] Specifically, the present invention defines earthquake monitoring time series data as a multivariate time series: , , represents the length of the earthquake time series monitored by the sensor, The earthquake monitoring data at each time point is dimensional vector, Represents the number of earthquake monitoring sensors, indicating the number of sensors from The recorded values of the sensors; at the same time, the tag information , ,like , it means that an earthquake has been detected at the current moment. , it means normal.
[0069] In a specific implementation of S12, according to the earthquake monitoring time series data, the current window and the prediction window are divided to construct a training set, wherein the training set includes a training window set and a training window true label set;
[0070] Specifically, this step may include the following sub-steps:
[0071] S21: According to the current sliding window size , prediction window size With moving step length Split the earthquake monitoring time series data to obtain the current window set With the prediction window set ,in Represents time The initial window size is The current window, Represents time The initial window size is The prediction window of
[0072] Specifically, from the window start point 0 to the window end point Slide and split the original long time series, where , the number of windows in each window set is +1, the window anomaly detection task involved in the present invention aims to detect anomalies in the current window. , determine the current window and its subsequent prediction window Whether it contains anomalies, and performs binary classification prediction for the window;
[0073] S22: According to the current sliding window size , prediction window size With moving step length Split the label data corresponding to the earthquake monitoring time series data to obtain the current window label set and the prediction window label set ;
[0074] Specifically, the same operation as S21 is performed on the label data to obtain the current window label set. and the prediction window label set For the true label, aggregate the entire window, , , set the entire window label as abnormal, otherwise it is normal. The advance window anomaly detection aims to detect the abnormality of the current window. Predict next window Tags .
[0075] S23: Set the current window With the prediction window set , Current window tab set and the prediction window label set Merge respectively to construct a training set, the training set includes a training window set and the training window label set ;
[0076] Specifically, and , and Merge them separately to get the training set , .
[0077] In a specific implementation of S13, based on the training set, the hidden layer representations of the current window and the prediction window are obtained by encoding through a continuous time self-attention encoder, and the current window label is obtained by decoding through a shared linear layer;
[0078] Specifically, Figure 2 As shown, the continuous-time self-attention encoder is composed of The single-layer encoding layer consists of a continuous time multi-head attention layer (CT-MHA), a residual connection and normalization layer (ADD&Norm) and a feed-forward fully connected network layer (Feed Forward). ADD&Norm helps solve the problem of gradient disappearance or explosion in deep networks and stabilize the training process; Feed Forward further processes data after self-attention to provide the model with nonlinear transformation capabilities. CT-MHA combines irregular time series resampling with multi-head attention, allowing the model to simulate the continuous dynamics of time series from a continuous perspective and capture the evolution of data-dependent processes in order to better encode data.
[0079] Specifically, the input linear layer is used to adjust the window time point dimension to In order to solve the problem of discontinuous embedding code in irregular time series, the current window is used as an example to perform the following specific processing:
[0080] Through the interpolation module (Interploate), the natural cubic curve interpolation is used to fit the earthquake monitoring time series data sequence in the current window and interpolate the time series data in the time set Resample up to get the query vector ,remember , for At time step The original time series is resampled by taking the value of , which effectively supports regular and irregular time series.
[0081] Use the first definite integral linear layer (ODELinear) and the second definite integral linear layer (ODELinear) to define ordinary differential equations for the key vector and value vector respectively. , ,in Respectively represent the parameters and No. The ordinary differential equation for the observation, and Represents the key vector and value vector The initial state of the ordinary differential equation for the observation, To calculate the inner product in the continuous time domain, define and , the value vector is defined as , given ,calculate arrive Expected value;
[0082] Multi-head continuous time attention ( ) layer consists of multiple continuous time attention ( ) head spliced together, ,in , , represents the timestamp in the time series after being processed by the interpolation module; specifically, , encoding sequence potential trajectory, capturing the temporal evolution relationship between observations, and obtaining continuous time hidden layer representation , , , , and is a parameter matrix used to transform the feature space dimension, , is the total number of attention heads;
[0083] The hidden layer representation of the current window is obtained through three layers of continuous first residual connection and normalization layer (ADD&Norm), first feedforward fully connected network layer (Feed Forward) and second residual connection and normalization layer (ADD&Norm). , ,in represents the batch size, Represents the size of the hidden layer dimension; similarly, the hidden layer representation of the prediction window Gather in time Carry out the above process to obtain.
[0084] Subsequently, the hidden layer representation of the current window is decoded through a shared linear layer to obtain the current window label ,in , , Indicates a linear transformation of the current window aggregate embedding to provide a suitable input for the binary classification output. Adjust the offset of the results to help learn nonlinear patterns in the data, ;
[0085] Specifically, the current window aggregate embedding , and Respectively express Average pooling and maximum pooling are performed in the time dimension, and the aggregated information of the entire window is added together, which not only retains the overall information of the window but also retains the significant features.
[0086] In a specific implementation of S14, the all-0 window is merged with the second half of the current window to form a merged window with the same length as the current window, the merged window is input into a self-attention decoder, the hidden layer representation of the current window is used to decode and generate a predicted hidden layer representation of the prediction window, and the prediction window label is obtained through a shared linear layer;
[0087] Specifically, the self-attention decoder includes several decoding layers, each decoding layer includes a second interpolation module, a self-attention layer, a third residual connection and normalization layer, a cross attention layer, a fourth residual connection and normalization layer, a second feedforward fully connected network layer, and a fifth residual connection and normalization layer. This step may include the following sub-steps:
[0088] S41: Set the second half of the current window to The part and size of The all-0 windows are spliced to obtain the merged window set , where the merge window ;
[0089] Specifically, the second half size of the current window is The part is used as the prompt information of the all-0 window.
[0090] S42: Through the second interpolation module, interpolate the non-zero time series to obtain the self-attention part query Vector, key With value , generating preliminary hidden representations as the query for the cross-attention part vector;
[0091] Specifically, Split and get a prompt window ,in Get non-zero time steps through the second interpolation module (Interploate) using natural cubic curve interpolation in time set Resample on the screen to get the resampling prompt window ,in . Splice to get the resampling merge window ,in , and then take a single window as an example to explain Perform linear changes and adjust the feature dimension to , embedded , where the linear layer weights , linear layer bias , linearly transform the embedding to obtain , and , , , and are the weights of their respective linear transformation layers. After the self-attention layer (Masked Self-Attention) , refers to the normalized exponential function, After that, the weights after the current time step need to be reset to 0 according to the causal mask to shield the information of future time steps, and the preliminary hidden layer representation is obtained through residual connection with random inactivation and layer normalization (ADD&Norm) .
[0092] Causal Mask , ensuring that the model can only see the data of the current time step and the time step before it when processing the window, preventing information leakage.
[0093] S43: Use the previous text Hidden layer representation as key for cross-attention layer With value , prediction decoding is taken Partially obtained ;
[0094] Specifically, here , , , express Batch processing form, similarly, after the cross attention layer (Cross Attention) , after residual connection with random loss and layer normalization (ADD&Norm), the preliminary decoding representation is obtained , through a feed-forward neural network (Feed Forward), and then pass through the residual connection with random inactivation and layer normalization (ADD&Norm) to obtain the output decoded embedding , after taking the time dimension Step as prediction output In this way, the prediction window information can be generated at one time according to the merged window set to realize encoding from the current window to the future window information.
[0095] S44: Decoding using shared linear layers to obtain prediction window labels ;
[0096] Specifically, , and Respectively express Average pooling and maximum pooling are performed in the time dimension. By sharing the linear layer, the window labels are predicted more uniformly, which promotes the exchange of information between the two parts of the model.
[0097] In a specific implementation of S15, based on the real label set of the training window, the label of the current window, and the label of the prediction window, a label loss is calculated, a distillation loss is calculated based on the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window, and the continuous time self-attention encoder, the self-attention decoder, and the shared linear layer are trained according to the label loss and the distillation loss;
[0098] Specifically, this step may include the following sub-steps:
[0099] S51: Use the cross entropy loss function to calculate the label loss between the current window true label and the current window label , and the difference between the true label of the prediction window and the label of the prediction window ,use Divergence calculates the distillation loss between the hidden representation of the prediction window and the predicted hidden representation of the prediction window ;
[0100] Specifically, , .
[0101] S52: Weighted calculation ,based on Back-propagation is performed on the continuous-time self-attention encoder and the shared linear layer, based on Back-propagate the self-attention decoder and shared linear layer using Optimize model parameters to train the model;
[0102] Specific, general invention , through multiple Back propagation is performed separately to achieve multi-task learning, and the current window anomaly detection task and the advance window anomaly detection task are optimized at the same time. At the same time, in order to allow the model to better learn how to encode the current window to obtain the next window information, knowledge distillation is performed based on the advance window embedding obtained by encoding the current window anomaly detection task, which is easier, to guide the model learning direction and accelerate convergence.
[0103] In a specific implementation of S16, earthquake monitoring data is acquired in real time, and the trained continuous-time self-attention encoder, self-attention decoder, and shared linear layer are used to predict earthquake conditions within a predetermined time in the future;
[0104] Specifically, the earthquake situation in the future predetermined time is set to be The all-0 prediction window is based on the acquired earthquake monitoring data, and the structure size is The current window is obtained through the process of steps S13-S14, thereby obtaining the prediction window label, that is, obtaining the earthquake situation within the future predetermined time.
[0105] It should be noted that in the process of real-time prediction, a test set needs to be constructed and the shared linear layer needs to be fine-tuned on the test set to adapt to data offset.
[0106] Specifically, in the online anomaly detection scenario, data prediction labels are continuously collected, which is essentially a binary classification task. The edge is calculated based on the predicted probability. ,in represents the probability of predicting a sample as abnormal. represents the probability of being predicted to be normal, , select the data with the smallest edge value and save it to the test set, while maintaining the balance of positive and negative samples, saving a certain amount of data space, according to The strategy replaces old data and fine-tunes and retrains the model's shared linear layer at fixed time intervals, taking into account efficiency while alleviating the model's poor performance in predicting data offset and samples that have never appeared.
[0107] Corresponding to the aforementioned embodiment of the supervised earthquake monitoring method for time series data, the present application also provides an embodiment of a supervised earthquake monitoring device for time series data.
[0108] Figure 3 is a block diagram of a supervised earthquake monitoring device for time series data according to an exemplary embodiment. Figure 3 , the device may include:
[0109] An acquisition module 21 is used to acquire earthquake monitoring time series data, wherein the earthquake monitoring time series data includes earthquake monitoring data at several time points and corresponding real label data;
[0110] A training set construction module 22 is used to divide the current window and the prediction window according to the earthquake monitoring time series data, so as to construct a training set, wherein the training set includes a training window set and a training window true label set;
[0111] A first hidden layer representation generation module 23 is used to obtain hidden layer representations of the current window and the prediction window based on the training set by encoding through a continuous time self-attention encoder, and obtain a current window label by decoding through a shared linear layer;
[0112] The second hidden layer representation generation module 24 is used to merge the all-0 window with the second half of the current window to form a merged window with the same length as the current window, input the merged window into the self-attention decoder, use the hidden layer representation of the current window to decode and generate the predicted hidden layer representation of the prediction window, and obtain the predicted window label through the shared linear layer;
[0113] A training module 25 is used to calculate a label loss based on the training window real label set, the current window label, and the predicted window label, calculate a distillation loss based on the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window, and train the continuous time self-attention encoder, the self-attention decoder, and the shared linear layer according to the label loss and the distillation loss;
[0114] The prediction module 26 is used to obtain real-time earthquake monitoring data and use the trained continuous-time self-attention encoder, self-attention decoder and shared linear layer to predict the earthquake situation within a predetermined time in the future.
[0115] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0116] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.
[0117] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the supervised earthquake monitoring method for time series data as described above. Figure 4 As shown, a hardware structure diagram of a supervised earthquake monitoring method for time series data provided by an embodiment of the present invention is provided for any device with data processing capability, except Figure 4 In addition to the processor, memory and network interface shown, any device with data processing capability in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capability, which will not be described in detail.
[0118] Accordingly, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by the processor, the supervised seismic monitoring method for time series data as described above is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be other external storage devices, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.
[0119] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.
[0120] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.
Claims
1. A supervised earthquake monitoring method for time series data, It is characterized in that include: Acquire earthquake monitoring time series data, wherein the earthquake monitoring time series data includes earthquake monitoring data at several time points and corresponding real label data; According to the earthquake monitoring time series data, the current window and the prediction window are divided to construct a training set, wherein the training set includes a training window set and a training window true label set; Based on the training set, the hidden layer representations of the current window and the prediction window are obtained by encoding through a continuous time self-attention encoder, and the current window label is obtained by decoding through a shared linear layer; Merge the all-0 window with the second half of the current window to form a merged window with the same length as the current window, input the merged window into the self-attention decoder, use the hidden layer representation of the current window to decode and generate the predicted hidden layer representation of the prediction window, and obtain the predicted window label through the shared linear layer; Based on the real label set of the training window, the label of the current window, and the label of the predicted window, the label loss is calculated, the distillation loss is calculated based on the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window, and the continuous time self-attention encoder, the self-attention decoder, and the shared linear layer are trained according to the label loss and the distillation loss; Real-time earthquake monitoring data is obtained, and the trained continuous-time self-attention encoder, self-attention decoder, and shared linear layer are used to predict earthquake conditions within a predetermined time in the future.
2. The method according to claim 1, It is characterized in that According to the earthquake monitoring time series data, the current window and the prediction window are divided to construct a training set, including: Based on the current window size , prediction window size With moving step length Split the earthquake monitoring time series data to obtain the current window set With the prediction window set ,in , Represents time The initial window size is The current window, Represents time The initial window size is The prediction window of According to the current sliding window size , prediction window size With moving step length Split the real label data corresponding to the earthquake monitoring time series data to obtain the real label set of the current window and the true label set of the prediction window ; Set the current window With the prediction window set , the current window real label set and the true label set of the prediction window Merge respectively to construct a training set, the training set includes a training window set and the training window true label set .
3. The method according to claim 1, It is characterized in that The continuous-time self-attention encoder includes several encoding layers, each encoding layer includes a first interpolation module, a first definite integral linear layer, a second definite integral linear layer, a continuous-time multi-head attention layer, a first residual connection and normalization layer, a first feedforward fully connected network layer and a second residual connection and normalization layer. For the input sequence data, the continuous-time self-attention encoder performs the following processing: The interpolation module fits the sequence data through natural cubic curve interpolation and resamples to obtain a first query vector; The first definite integral linear layer and the second definite integral linear layer calculate the first key vector and the first value vector respectively; The continuous-time multi-head attention layer captures the time evolution relationship between the observations according to the first query vector, the first key vector and the first value vector to obtain a continuous-time hidden layer representation; The continuous-time hidden layer representation obtains the hidden layer representation of the sequence data through a continuous first residual connection and normalization layer, a first feedforward fully connected network layer and a second residual connection and normalization layer.
4. The method according to claim 1, It is characterized in that Use the shared linear layer to decode the hidden layer representation of the current window to get the current window label ,in , and Represent average pooling and maximum pooling respectively, is the hidden layer representation of the current window, Indicates a linear transformation of the current window aggregate embedding, The offset to adjust the result by.
5. The method according to claim 1, It is characterized in that The self-attention decoder includes several decoding layers, each of which includes a second interpolation module, a self-attention layer, a third residual connection and normalization layer, a cross attention layer, a fourth residual connection and normalization layer, a second feedforward fully connected network layer, and a fifth residual connection and normalization layer. For the input sequence data, the self-attention decoder performs the following processing: For non-0 segment data in the sequence data, the second interpolation module performs fitting and resampling through natural cubic curve interpolation and splicing with the 0 segment data in the sequence, and obtains a second query vector, a second key vector and a second value vector through embedding and linear transformation; A preliminary hidden layer representation is obtained through a third residual connection and a normalization layer as a third query vector; Based on the hidden layer representation of the current window, a third key vector and a third value vector are obtained, which are decoded through a cross attention layer, and passed through a fourth residual connection and normalization layer, a second feedforward fully connected network layer, and a fifth residual connection and normalization layer to obtain a predicted hidden layer representation corresponding to the sequence data.
6. The method according to claim 1, It is characterized in that Based on the real label set of the training window, the label of the current window, and the label of the predicted window, the label loss is calculated, the distillation loss is calculated based on the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window, and the continuous time self-attention encoder, the self-attention decoder, and the shared linear layer are trained according to the label loss and the distillation loss, including: Calculate the first label loss , as the first loss; among them represents the cross entropy loss function, is the real label of the current window, The current window label; Calculate the second label loss ,in is the true label of the prediction window, is the prediction window label; Calculating distillation losses ,in is the prediction window size, represents the KL divergence, refers to the normalized exponential function, They represent the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window respectively; A second loss is obtained by weighted summing the second label loss and the distillation loss; The continuous-time self-attention encoder and the shared linear layer are back-propagated based on the first loss, and the self-attention decoder and the shared linear layer are back-propagated based on the second loss, thereby achieving training.
7. The method according to claim 1, It is characterized in that During the real-time prediction process, a test set is constructed and the shared linear layer is fine-tuned on the test set. The construction process of the test set is specifically as follows: Calculate the margin based on the predicted probability ,in represents the probability of being predicted as an anomaly, It represents the probability of being predicted as normal, and the data with the smallest marginal value is selected and saved into the test set, where the size of the test set is constant, and the data therein is replaced according to the first-in-first-out strategy.
8. A supervised earthquake monitoring device for time series data, It is characterized in that include: An acquisition module is used to acquire earthquake monitoring time series data, wherein the earthquake monitoring time series data includes earthquake monitoring data at several time points and corresponding real label data; A training set construction module, used to divide the current window and the prediction window according to the earthquake monitoring time series data, so as to construct a training set, wherein the training set includes a training window set and a training window true label set; A first hidden layer representation generation module is used to obtain hidden layer representations of the current window and the prediction window based on the training set by encoding through a continuous time self-attention encoder, and obtain a current window label by decoding through a shared linear layer; The second hidden layer representation generation module is used to merge the all-0 window with the second half of the current window to form a merged window with the same length as the current window, input the merged window into the self-attention decoder, use the hidden layer representation of the current window to decode and generate the predicted hidden layer representation of the prediction window, and obtain the predicted window label through the shared linear layer; A training module, used to calculate the label loss based on the real label set of the training window, the current window label, and the predicted window label, calculate the distillation loss based on the hidden layer representation of the prediction window and the predicted hidden layer representation of the prediction window, and train the continuous time self-attention encoder, self-attention decoder and shared linear layer according to the label loss and distillation loss; The prediction module is used to obtain real-time earthquake monitoring data and use the trained continuous-time self-attention encoder, self-attention decoder and shared linear layer to predict earthquake conditions within a predetermined time in the future.
9. An electronic device, It is characterized in that include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having computer instructions stored thereon, It is characterized in that When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.