An event sequence prediction method based on time series convolution and relational modeling

Abstract

The present invention discloses an event sequence prediction method based on time series convolution and relational modeling, comprising: step 1, obtaining an event sequence training set from a database; step 2, preprocessing the original data; step 3, extracting features from the tag information in the historical sequence using a tag feature encoder; step 4, extracting features from the time information in the historical sequence using a time series feature encoder; step 5, fusing the tag feature coding and time feature coding of historical events, and outputting a feature representation for a single historical event; step 6, constructing a time series correlation graph between events based on the event coding, and outputting the feature coding of the historical sequence; step 7, calculating the conditional strength for each type of event; step 8, calculating the model loss function and updating the parameters; step 9, judging whether the loss curve of the model converges, and returning to step 8 if not; and step 10, saving the trained model and deploying it.

Description

Technical Field

[0001] The present invention relates to an event sequence prediction method, in particular to an event sequence prediction method based on time series convolution and relational modeling. Background Art

[0002] Event sequence prediction is a key research area in time series analysis, with scholars conducting research in this field as early as the 1970s. Event sequence data is ubiquitous in our daily lives, and event sequence prediction techniques have been applied across numerous scientific fields, including social sciences, medicine, geology, and physics. Analyzing and understanding event sequence data and accurately predicting future events have significant social value, and as such, this field has garnered widespread attention from the academic community.

[0003] Conventional event sequence data is a set of sequences of events arranged in chronological order. Each event sample contains a timestamp and additional tag information. The goal of event sequence prediction is to use historical event sequences to predict information about future events. Current approaches to this problem fall into two main categories: traditional machine learning-based methods and deep learning-based methods.

[0004] Traditional machine learning methods are early approaches to event sequence prediction and can be broadly categorized into two types: Markov model-based methods and point process model-based methods. Markov model-based methods construct a system of non-deterministic patterns using an undirected graph and establish the dependency relationship between the current event and n historical events through the nth-order Markov property. Point process model-based methods, on the other hand, construct a stochastic process model based on the historical sequence, allowing for a more natural modeling of the dependencies between different events in the sequence. The core of these methods is modeling conditional intensity functions. For example, the Hawkes point process model proposed by Hawkes et al. uses conditional intensity functions to describe the incentive process of historical events on future events and considers the influence of the underlying strength of the environment, resulting in excellent predictive performance. Reference: Chen J, Hawkes AG, Scalas E, et al: Performance of information criteria for selecting Hawkes process models of financial data. In: Quantitative Finance, 2018: 225-235.

[0005] In recent years, due to the increase in device computing power and the continuous development of artificial intelligence technology, deep learning-based methods have gradually become the mainstream approach in this field. Currently, deep learning-based event sequence prediction models are mainly designed based on recurrent neural networks. Because deep learning models can automatically learn complex high-order features in data, this type of method is capable of processing large-scale data. A typical method is the RMTPP model proposed by Nan Du et al., which uses long short-term memory networks and gated recurrent units to encode event sequence features and simultaneously considers the impact of historical event sequences, current events, and base strength on future events, significantly improving model performance. Reference: Du N, Dai H, Trivedi R, et al: Recurrent marked temporal point processes: Embedding event history to vector. In: 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016: 1555-1564.

[0006] Although models based on recurrent neural networks have achieved a certain level of predictive accuracy, their inherent structural characteristics make parallel computing difficult, resulting in low computational efficiency during model training and inference. Furthermore, while recurrent neural networks can encode and represent sequential data, they cannot directly describe the influence relationships between different events in the sequence. Consequently, these models not only lack interpretability but also fail to fully explore and analyze the influence relationships between events, limiting model performance. Summary of the Invention

[0007] Purpose of the invention: The technical problem to be solved by the present invention is to provide an event sequence prediction method based on time series convolution and relational modeling in response to the shortcomings of the existing technology.

[0008] In order to solve the above technical problems, the present invention discloses an event sequence prediction method based on time series convolution and relational modeling, comprising the following steps:

[0009] Step 1: Generate a historical event sequence dataset D for model training from the database train , that is, the original training data set D train ,Each event data contains the timestamp and tag information of the historical event, and builds an event sequence prediction model;

[0010] Step 2: Preprocess the original training dataset, including data cleaning and event interval calculation. The data cleaning process includes removing invalid samples, patching abnormal samples, and normalizing the data.

[0011] Step 3: Use the marker feature encoder f mark For the original training data set D train The tag information of historical events is extracted to obtain the tag feature code v mark ;

[0012] Step 4: Use the temporal feature encoder f time For the original training data set D train The timestamp information of historical events is extracted to obtain the time feature code v time ;

[0013] Step 5: Encode the marker features v mark and temporal feature encoding v time Perform feature fusion to obtain the feature representation v for a single event event ;

[0014] Step 6: Based on the feature coding of a single historical event in the historical event sequence, a temporal correlation graph between events is constructed, and the temporal correlation graph is used to represent the feature of the historical event sequence to obtain the sequence feature s i ;

[0015] Step 7, using sequence feature s i Calculate the conditional strength function λ for each type of event, and use conditional strength reasoning to obtain the timestamp and tag information of future events;

[0016] Step 8: Calculate the loss function value of the event sequence prediction model, calculate the corresponding gradient, and use the optimization algorithm to implement the back propagation of the event sequence prediction model to update the parameters of the event sequence prediction model;

[0017] Step 9: Determine whether the loss curve of the event sequence prediction model has converged. If not, return to step 8 to continue parameter optimization of the event sequence prediction model.

[0018] Step 10: Complete the event sequence prediction based on time series convolution and relational modeling, save the trained event sequence prediction model, and deploy the event sequence prediction model to the server.

[0019] In step 1 of the present invention, a historical event sequence data set D for model training is generated from the database. train , which contains N tHistorical event data, each historical event data includes the timestamp information of the historical event and the tag information corresponding to the event; the sequence arranges the events, and the codes corresponding to the events increase from 0; based on the training data set, an event sequence prediction model is constructed.

[0020] In step 2 of the present invention, the historical event sequence data set generated in step 1 is input as a training set, and the original training data set is cleaned and the event interval time is calculated;

[0021] The data cleaning steps include: train Invalid and duplicate samples were counted and deleted; the local outlier factor algorithm (reference: Yang J, Zhong N, Yao Y, et al.: Local peculiarity factor and its application in outlier detection. In: Acm Sigkdd International Conference on Knowledge Discovery & Data Mining, 2008: 776-784) was used to screen D train The outliers in the dataset are corrected using the average value of the historical event sequence. The maximum and minimum normalization method is used to normalize the data of the training set samples. The data is divided into batches and batch normalized using the Batch Normalization method (reference: Ioffe S, Szegedy C: Batchnormalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, 2015: 448-456.).

[0022] Step 3 of the present invention comprises:

[0023] Step 3-1, define a temporal attention convolutional network as the tag feature encoder f mark, using the linear rectification function ReLU (reference: Glorot X, Bordes A, Bengio Y: Deep Sparse Rectifier Neural Networks. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. 2011: 315-323.) as the activation function of the network, and using the Kaiming Initialization method (reference: He K, Zhang X, Ren S, et al: Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In: International Conference on Computer Vision 2015: 1026—1034.) to initialize the network parameters;

[0024] Step 3-2, for D train Calculate the temporal correlation matrix between different historical event tag information in the corresponding tag information sequence:

[0025]

[0026]

[0027] Among them, m 1:T It represents the sequence of label information corresponding to the 1st to Tth events in the historical sequence, that is, the label information sequence, k 1:T and q 1:T Indicates m 1:T The corresponding key values ​​and query, and represents the linear mapping layer used to calculate keys and queries, i and j represent historical event steps, k i and q j Represents the keys and query corresponding to the event step, d k represents the dimension of the feature vector, W i,j represents the general autocorrelation matrix of the series, Wt i,j Represents the temporal correlation matrix of the sequence;

[0028] Step 3-3, use the temporal attention structure to train The tag information sequence m corresponding to the historical event sequence1:T Perform preliminary feature extraction; the calculation process of temporal attention TA is as follows:

[0029]

[0030] Among them, v 1:T is m 1:T The corresponding re-encoded tensor value values, The linear mapping layer used to calculate values, softmax is a normalized exponential function, Preliminary feature encoding of the tag information output by the temporal attention structure;

[0031] Steps 3-4, in Based on the temporal convolution structure, D train The tag information sequence corresponding to the historical event sequence is further extracted for feature extraction. The calculation process of the temporal convolution TC is as follows:

[0032]

[0033] Among them, Conv cau and Conv dil Represent causal convolution and dilated convolution respectively, and v mark Represents the output of two convolutions, v mark Encode the marker features of the obtained historical events.

[0034] In step 4 of the present invention, a nonlinear mapping layer is defined As the temporal feature encoder f time , using ReLU as the activation function of the network, and using Kaiming initialization method to initialize the network parameters; using To D train The timestamp information of historical events is extracted to obtain the time feature code v time .

[0035] In step 5 of the present invention, the marking feature is encoded v mark and temporal feature encoding v time Perform feature fusion. The feature fusion process is as follows:

[0036]

[0037] in, and is the linear mapping layer used for fusion, v event is the obtained single event feature representation.

[0038] Step 6 of the present invention comprises:

[0039] Step 6-1, use the sliding window method (reference: Gemulla R, Lehner W: Sampling time-based sliding windows in bounded space. In: Acm Sigmod International Conference on Management of Data. 2008: 379) to train For each event sequence, define L w is the window size, in v event Based on this, a window-based event feature sequence s is generated ori , and construct a time series correlation graph on each window;

[0040] Step 6-2, for the window-based event feature sequence s ori , calculate the correlation weight a between different events in the sequence i,j , the method is:

[0041]

[0042] Among them, exp represents the exponential function, and represents the feature sequence of the corresponding event step, w rel Represents the weight of the neural network, Ne i Represents the neighbor node of the event in the time series correlation graph, k represents the sequence number of the neighbor node, and ReLU is the linear rectification function;

[0043] Step 6-3, construct the adjacency matrix A of the time series correlation graph using correlation weights rel , and aggregate node information on the time series correlation graph to recode and splice event nodes to obtain the sequence characteristics of historical events s i . (Reference: Bahdanau D, Cho K, Bengio Y: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations. 2015)

[0044] In step 7 of the present invention, a conditional intensity function λ is defined for each type of event, and the sequence characteristics s of historical events are used. i Calculate λ for different types of events, i.e. λ k (t), the method is as follows:

[0045] λ k (t) = exp(w v ·s i +w d ·(tt j )+b e )

[0046] Among them, w v and w d represents the weight of the neural network, b e represents the basic intensity, t represents the current time, t j Represents the timestamp of the previous event; uses the intensity function to output prediction information for future events (reference: Xiao S, Yan J, Chu SM, et al: Modeling The Intensity Function Of Point Process Via Recurrent NeuralNetworks. In: Proceedings of the 35th Conference on ArtificialIntelligence. 2017: 1597—1603.), including the timestamp and tag information

[0047] In step 8 of the present invention, the log-likelihood function is defined as the loss function of the model, and the model is calculated in D train The loss function value on is used to calculate the corresponding gradient, and the Adam optimization algorithm (reference: Kingma DP, Ba J. Adam: A method for stochastic optimization. In: 3rd International Conference on Learning Representations. 2015.) is used to implement the back propagation of the model and optimize the model parameters.

[0048] In step 10 of the present invention, the trained model is saved, deployed to the server, and an interface service is provided externally.

[0049] Beneficial effects:

[0050] 1. The present invention designs a temporal attention convolutional network, which simulates the sequence modeling process of recurrent neural networks through the causal convolutional network in the temporal convolution, and the calculation process can be parallelized, so the computational efficiency of the model during training and inference is significantly improved.

[0051] 2. Since the relevant attention mechanism is designed in the temporal attention convolutional network proposed in this invention, the model can establish the dependency between the current event and earlier historical events, expand the receptive field of the model, and enrich the expressive ability of the model.

[0052] 3. The present invention proposes a method for directly modeling the influence relationship between events for event sequence related data by constructing a time series correlation graph, which makes the model interpretable and improves the model's predictive performance for future events. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0054] Figure 1 It is a schematic diagram of the process of the present invention.

[0055] Figure 2 This is the overall computational framework diagram of the event sequence prediction method proposed in this invention.

[0056] Figure 3 This is the model structure diagram of the temporal attention convolutional network proposed in this invention.

[0057] Figure 4 This is a schematic diagram of the test results of the example verification of the method proposed in the present invention on two event sequence prediction data sets. DETAILED DESCRIPTION

[0058] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0059] like Figure 1 As shown in Figure 1, an event sequence prediction method based on time series convolution and relational modeling includes 10 steps:

[0060] In step 1, data sampling is performed in the existing database according to the size of the database to generate the event sequence dataset D for model training. train , which contains N t The data consists of a set of recorded historical events, each of which includes the timestamp of the event and the tag information corresponding to the event. The events are arranged and the corresponding codes are incremented starting from 0. Based on the training data set, an event sequence prediction model is constructed.

[0061] In step 2, the currently generated event sequence data is input into the algorithm as a training set, and the original data set is cleaned and the event interval time is calculated. The data cleaning step includes train Statistics and deletion of invalid and duplicate samples; use local outlier factor algorithm to screen D trainThe outliers in the dataset are corrected using the mean value, and the training set samples are normalized using the maximum and minimum normalization method. The data is divided into batches and batch normalization is performed using the Batch Normalization method.

[0062] Step 3 includes the following steps:

[0063] Step 3-1, define a temporal attention convolutional network as the tag feature encoder f mark , ReLU is used as the activation function of the network, and the Kaiming initialization method is used to initialize the network parameters.

[0064] Step 3-2, for D train For a historical event sequence, calculate the temporal correlation matrix between different event marker information in its corresponding marker information sequence:

[0065]

[0066]

[0067]

[0068] where m 1:T represents the sequence of label information corresponding to the 1st to Tth events in the historical sequence, k 1:T and q 1:T Indicates m 1:T Corresponding keys and queries, and Indicates the linear mapping layer used to calculate keys and queries, d k represents the dimension of the feature vector, W i,j represents the general autocorrelation matrix of the series, Wt i,j Represents the temporal correlation matrix of the sequence.

[0069] Step 3-3, use the temporal attention structure to train The tag information sequence m corresponding to the historical event sequence in the CNN is used for preliminary feature extraction. The calculation process of temporal attention TA is as follows:

[0070]

[0071] where v 1:T is m 1:T The corresponding re-encoded values, The linear mapping layer used to calculate values, softmax is a normalized exponential function, Preliminary feature encoding of the tag information output by the temporal attention structure.

[0072] Steps 3-4, in Based on the temporal convolution structure, D train The tag information sequence corresponding to the historical event sequence is further extracted for feature extraction. The calculation process of the temporal convolution TC is as follows:

[0073]

[0074] Among them, Conv cau and Conv dil Represent causal convolution and dilated convolution respectively, and v mark Represents the output of two convolutions, v mark Encode the obtained historical event marker features.

[0075] In step 4, we define the nonlinear mapping layer As the temporal feature encoder f time , using ReLU as the activation function of the network and using Kaiming initialization method to initialize the network parameters. To D train The timestamp information of historical events is extracted to obtain the time feature code v time .

[0076] In step 5, we mark and v time Perform feature fusion. The feature fusion process is as follows:

[0077]

[0078] in and is the linear mapping layer used for fusion, v event is the obtained single event feature representation.

[0079] Step 6 includes the following steps:

[0080] In step 6-1, we use the sliding window method to train For each event sequence, define L w is the window size, in v event Based on this, a window-based event feature sequence s is generated ori , and construct a temporal correlation graph on each window.

[0081] Step 6-2, for s ori , we calculate the correlation weights between different events in the sequence:

[0082]

[0083] where w rel Represents the weight of the neural network, Ne i Represents the neighbor nodes of an event in the temporal correlation graph.

[0084] In step 6-3, we use the correlation weights to construct the adjacency matrix A of the time series correlation graph rel , and aggregate node information on the time series correlation graph to recode and splice event nodes to obtain sequence features s i .

[0085] In step 7, we define the conditional intensity function λ for each type of event and use the historical sequence feature s i Compute λ for different types of events:

[0086] λ k (t) = exp(w v ·s i +w d ·(tt j )+b e )

[0087] where w v and w d represents the weight of the neural network, b e represents the basic strength, t represents the current time, t j Represents the timestamp of the previous event. Use the intensity function to output prediction information for future events, including timestamps and tag information

[0088] In step 8, we define the log-likelihood function as the loss function of the model and calculate the model's loss in D train The loss function value on is used to calculate the corresponding gradient, and the Adam optimization algorithm is used to implement the back propagation of the model and optimize the model parameters.

[0089] In step 9, we determine whether the loss curve of the model has converged. If the curve has not converged, we return to step 8 to continue optimizing the model parameters.

[0090] In step 10, we save the trained model and deploy it to the server, providing RESTful API interface services.

[0091] Example

[0092] To verify the effectiveness of the present invention, we conducted practical verification on two event sequence prediction task datasets collected in real scenarios, including the IPTV dataset and the MIMIC-II dataset. The IPTV dataset is event sequence data of users' cable TV program viewing behavior provided by China Telecom, containing viewing behavior sequences of 2,967 users; the MIMIC-II dataset is event sequence data related to medical diagnosis of patients provided by a medical center, recording the medical treatment data of 53,423 patients who visited the medical center between 2001 and 2008. This embodiment now takes an event sequence data used for testing in the IPTV dataset as an example and performs event prediction according to the following steps:

[0093] 1. Perform data preprocessing on the event sequence data, including data cleaning and interval calculation. The data cleaning process includes removing invalid samples, fixing abnormal samples, and normalizing the data.

[0094] 2. Using the marker feature encoder f mark The feature extraction of the tag information of the historical events in the event sequence data is performed to obtain the tag feature code v mark .

[0095] 3. Using the temporal feature encoder f time The time stamp information of the historical events in the event sequence data is extracted to obtain the time feature code v time .

[0096] 4. Use v obtained in step 3 mark and v time Perform feature fusion to obtain the feature representation v for a single event event .

[0097] 5. Using the feature coding of a single event in the historical sequence obtained in step 4, construct a temporal correlation graph between events, and use it to represent the features of the historical sequence to obtain the sequence feature s i .

[0098] 6. Use the historical sequence feature s output in step 5 i Calculate the conditional strength function λ for each type of event, use the conditional strength to infer the timestamp and tag information of future events, and output the prediction results.

[0099] Our method achieved a prediction accuracy of 72.21% on the IPTV dataset, with a root mean square error (RMSE) of 12.632; and 84.33% on the MIMIC-II dataset, with an RMSE of 1.982. This improvement in prediction accuracy compared to existing methods demonstrates the effectiveness of our proposed event sequence prediction method based on time series convolution and relational modeling.

[0100] like Figure 2 As shown, the overall computational framework diagram of the event sequence prediction algorithm proposed in the present invention for samples is shown. It can be clearly observed that our event prediction algorithm is divided into three stages. The first stage is to perform feature encoding for each event in the historical sequence, including feature encoding for tag information and feature encoding for timestamps; the second stage is feature encoding for the entire historical sequence; the third stage is to use the feature representation of the historical sequence to predict relevant information of future events. In the first stage, we use the designed temporal attention convolutional network to feature encode the tag information of historical events, and use the nonlinear mapping layer to feature encode the timestamps of historical events; in the second stage, we construct a temporal correlation graph on the historical sequence, and use this graph structure to realize the encoding of the historical sequence; in the third stage, we define and calculate the conditional strength of each type of event, and predict future events based on the conditional strength.

[0101] like Figure 3 As shown in the figure, the specific model structure diagram of the temporal attention convolutional network proposed in the present invention is shown. Through this figure, we can find that the temporal attention convolutional network is mainly constructed by three modules, namely the temporal attention module (Temporal Attention), the temporal convolution module (Temporal Convolution) and the enhanced residual module (Enhanced Residual). Among them, the temporal attention module and the temporal convolution module realize efficient feature extraction for the input sequence, and the role of the enhanced residual module is to use the weights obtained in the temporal attention module to increase the differences between different sequence points in the input sequence features and accelerate the convergence speed of the model in the training stage.

[0102] like Figure 4The table shows the test results of the proposed method on two event sequence prediction datasets (IPTV dataset and MIMIC-II dataset). The proposed method is compared with existing methods, comparing prediction accuracy and RMSE. The first column of the table lists the different datasets, the second column lists the existing methods for comparison, and the last row for each dataset shows the proposed method. The comparison methods include the RMTPP model (reference: Du N, Dai H, Trivedi R, et al: Recurrent marked temporal point processes: Embedding event history to vector. In: 22th ACM SIGKDD international conference on knowledge discovery and data mining. 2016: 1555-1564.), the Intensity-RNN model (reference: Xiao S, Yan J, Chu SM, et al: Modeling The Intensity Function Of PointProcess Via Recurrent Neural Networks. In: Proceedings of the 35th Conference on Artificial Intelligence. 2017: 1597—1603.), the NHP model (reference: M, Charpentier B, Günnemann S: Uncertainty on asynchronous time event prediction. In: Advances in Neural Information Processing Systems. 2019, 32.), FullyNN-TPP (reference Omi T, Aihara K: Fully neural network based model for general temporal point processes. In: Advances in neural information processing systems, 2019.) and SAHP model (reference: Zhang Q, Lipani A, Kirnap O, et al: Self-attentive Hawkes process. In: International conference on machine learning. PMLR. 2020: 11183-11193.). The test results show that the present invention is superior to the compared models in both prediction accuracy and RMSE, proving the feasibility and effectiveness of this method of using temporal attention convolution for sequence feature extraction and using temporal correlation graphs for modeling the relationship between different events in historical sequences.

[0103] The present invention provides a method and approach for event sequence prediction based on time series convolution and relational modeling. There are numerous methods and approaches for implementing this technical solution. The foregoing is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. An event sequence prediction method based on time series convolution and relational modeling, characterized in that: The following steps are involved: Step 1: Generate a historical event sequence dataset D for model training from the database train , that is, the original training data set D train ,Each event data contains the timestamp and tag information of the historical event, and builds an event sequence prediction model; Step 2: Preprocess the original training dataset, including data cleaning and event interval calculation. The data cleaning process includes removing invalid samples, patching abnormal samples, and normalizing the data. Step 3: Use the marker feature encoder f mark For the original training data set D train The tag information of historical events is extracted to obtain the tag feature code v mark ; Step 4: Use the temporal feature encoder f time For the original training data set D train The timestamp information of historical events is extracted to obtain the time feature code v time ; Step 5: Encode the marker features v mark and temporal feature encoding v time Perform feature fusion to obtain the feature representation v for a single event event ; Step 6: Based on the feature coding of a single historical event in the historical event sequence, a temporal correlation graph between events is constructed, and the temporal correlation graph is used to represent the feature of the historical event sequence to obtain the sequence feature s i ; Step 7, using sequence feature s i Calculate the conditional strength function λ for each type of event, and use conditional strength reasoning to obtain the timestamp and tag information of future events; Step 8: Calculate the loss function value of the event sequence prediction model, calculate the corresponding gradient, and use the optimization algorithm to implement the back propagation of the event sequence prediction model to update the parameters of the event sequence prediction model; Step 9: Determine whether the loss curve of the event sequence prediction model has converged. If not, return to step 8 to continue parameter optimization of the event sequence prediction model. Step 10: Complete the event sequence prediction based on time series convolution and relational modeling, save the trained event sequence prediction model, and deploy the event sequence prediction model to the server.

2. The event sequence prediction method based on time series convolution and relational modeling according to claim 1, characterized in that: In step 1, a historical event sequence dataset D for model training is generated from the database train , which contains N t Historical event data, each historical event data includes the timestamp information of the historical event and the tag information corresponding to the event; the sequence arranges the events, and the codes corresponding to the events increase from 0; based on the training data set, an event sequence prediction model is constructed.

3. The event sequence prediction method based on time series convolution and relational modeling according to claim 2, characterized in that: In step 2, the historical event sequence dataset generated in step 1 is used as the training set input, and the original training dataset is cleaned and the event interval time is calculated; The data cleaning steps include: train Invalid and duplicate samples are counted and deleted; the local outlier factor algorithm is used to screen D train The outliers in the dataset are corrected using the average value of the historical event sequence; the maximum and minimum normalization method is used to normalize the data of the samples in the training set, the data are sorted into batches, and the BatchNormalization method is used to normalize the data.

4. The event sequence prediction method based on time series convolution and relational modeling according to claim 3, characterized in that: Step 3 includes: Step 3-1, define a temporal attention convolutional network as the tag feature encoder f mark , use the linear rectification function ReLU as the activation function of the network, and use the Kaiming Initialization method to initialize the network parameters; Step 3-2, for D train Calculate the temporal correlation matrix between different historical event tag information in the corresponding tag information sequence: Among them, m 1:T It represents the sequence of label information corresponding to the 1st to Tth events in the historical sequence, that is, the label information sequence, k 1:T and q 1:T Indicates m 1:T The corresponding key values ​​and query, and represents the linear mapping layer used to calculate keys and queries, i and j represent historical event steps, k i and q j Represents the keys and query corresponding to the event step, d k represents the dimension of the feature vector, W i,j represents the general autocorrelation matrix of the series, Wt i,j Represents the temporal correlation matrix of the sequence; Step 3-3, use the temporal attention structure to train The tag information sequence m corresponding to the historical event sequence 1:T Perform preliminary feature extraction; the calculation process of temporal attention TA is as follows: Among them, v 1:T is m 1:T The corresponding re-encoded tensor value values, The linear mapping layer used to calculate values, softmax is a normalized exponential function, Preliminary feature encoding of the tag information output by the temporal attention structure; Steps 3-4, in Based on the temporal convolution structure, D train The tag information sequence corresponding to the historical event sequence is further extracted for feature extraction. The calculation process of the temporal convolution TC is as follows: Among them, Conv cau and Conv dil Represent causal convolution and dilated convolution respectively, and v mark Represents the output of two convolutions, v mark Encode the marker features of the obtained historical events.

5. The event sequence prediction method based on time series convolution and relational modeling according to claim 4, characterized in that: In step 4, define the nonlinear mapping layer As the temporal feature encoder f time , using ReLU as the activation function of the network, and using Kaiming initialization method to initialize the network parameters; using To D train The timestamp information of historical events is extracted to obtain the time feature code v time .

6. The event sequence prediction method based on time series convolution and relational modeling according to claim 5, characterized in that: In step 5, the marker feature is encoded v mark and temporal feature encoding v time Perform feature fusion. The feature fusion process is as follows: in, and is the linear mapping layer used for fusion, v event is the obtained single event feature representation.

7. The event sequence prediction method based on time series convolution and relational modeling according to claim 6, characterized in that: Step 6 includes: Step 6-1, use the sliding window method to train For each event sequence, define L w is the window size, in v event Based on this, a window-based event feature sequence s is generated ori , and construct a time series correlation graph on each window; Step 6-2, for the window-based event feature sequence s ori , calculate the correlation weight a between different events in the sequence i,j , the method is: Among them, exp represents the exponential function, and represents the feature sequence of the corresponding event step, w rel Represents the weight of the neural network, Ne i Represents the neighbor node of the event in the time series correlation graph, k represents the sequence number of the neighbor node, and ReLU is the linear rectification function; Step 6-3, construct the adjacency matrix A of the time series correlation graph using correlation weights rel , and aggregate node information on the time series correlation graph to recode and splice event nodes to obtain the sequence characteristics s of historical events i .

8. The event sequence prediction method based on time series convolution and relational modeling according to claim 7, characterized in that: In step 7, a conditional intensity function λ is defined for each type of event, and the sequence characteristics s of historical events are used i Calculate λ for different types of events, i.e. λ k (t), the method is as follows: λ k (t)=exp(w v ·s i +w d ·(t-t j )+b e ) Among them, w v and w d represents the weight of the neural network, b e represents the basic strength, t represents the current time, t j Represents the timestamp of the previous event; uses the intensity function to output prediction information for future events, including timestamps and tag information 9. The event sequence prediction method based on time series convolution and relational modeling according to claim 8, characterized in that: In step 8, define the log-likelihood function as the loss function of the model and calculate the model's D train The loss function value on is used to calculate the corresponding gradient, and the Adam optimization algorithm is used to implement the back propagation of the model and optimize the model parameters.

10. The event sequence prediction method based on time series convolution and relational modeling according to claim 9, characterized in that: In step 10, the trained model is saved, deployed to the server, and provided with external interface services.

Images