Horx process modeling method, device and equipment based on Mama architecture and medium

Abstract

The invention discloses a Mamba architecture-based Horx process modeling method, device and equipment and a medium, and the method comprises the steps: building a training data set which comprises a plurality of event sequence data; generating a hidden state representation based on the selective state space model and the event sequence data; based on the hidden state representation, establishing an intensity function under the hox process; and sequentially inputting the event sequence data into the intensity function under the Horkes process for iterative training until a preset convergence condition is met, and obtaining the trained intensity function under the Horkes process as an event sequence prediction model. A selective state space mechanism is applied to complex event sequence modeling, an efficient and dynamic solution is provided, long-distance dependence can be flexibly captured, the dynamic relation between complex events can be accurately modeled, and the modeling efficiency is improved. Effective methods and tools are provided for prediction, analysis and management of complex event sequence type data such as medical electronic medical records, social media activity analysis and financial transactions.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a Hawkes process modeling method, device, equipment and medium based on the Mamba architecture. Background Art

[0002] Asynchronous and irregular event sequences are widely present in fields such as social media user behavior, financial market transactions, and medical records. A notable feature of these event sequences is that the time intervals between events are uneven, breaking the assumption of equal intervals in traditional time series. In addition, the dependencies between events are complex and diverse, and may involve a variety of dynamic features and statistical relationships, including short-term and long-term dependencies, mutual influences between multiple event types, and time-varying and context-dependent characteristics that change with time or external conditions. This complexity poses a huge challenge to traditional modeling methods.

[0003] At present, the modeling methods for asynchronous event sequences are mainly divided into traditional statistical models and neural network-based methods, but these methods still have many limitations when dealing with complex event sequence data. Among them, statistical point process models such as the Hawkes process can capture the self-excitation effect of events, but they perform poorly in dealing with mutual inhibition and nonlinear dependencies of events, and have limited capabilities in dealing with high-dimensional data and complex time dynamics. In contrast, point process modeling methods based on recurrent neural networks (RNN and its variants LSTM, GRU), although they can enhance flexibility, still face problems such as gradient vanishing or gradient exploding when dealing with long-term dependencies. Similarly, although the Transformer Hawkes process performs well in modeling short sequence data, the computational complexity of its self-attention mechanism is too high on long sequences, limiting its ability to capture long-distance dependencies. Summary of the invention

[0004] Based on this, it is necessary to provide a Hawkes process modeling method, device, computer equipment and storage medium based on the Mamba architecture to address the above technical problems, so as to solve at least one problem existing in the above-mentioned prior art.

[0005] In a first aspect, the embodiment of the present application is implemented by providing a Hawkes process modeling method based on the Mamba architecture, comprising the following steps: Establishing a training data set, wherein the training data set includes a plurality of event sequence data; generating a hidden state representation based on the selective state space model and the event sequence data; Based on the hidden state representation, an intensity function under the Hawkes process is established; The event sequence data are sequentially input into the intensity function under the Hawkes process for iterative training until a preset convergence condition is met, thereby obtaining a trained intensity function under the Hawkes process as an event sequence prediction model.

[0006] In one embodiment, generating a hidden state representation based on the selective state space model and the event sequence data includes: Inputting each data in each event sequence data into the selective state space model in turn for iteration, and updating the hidden state based on the state matrix output by the selective state space model during each iteration; The hidden states updated in each iteration are concatenated according to the timestamps to obtain multiple output sequences, and all the output sequences are fitted to obtain output state representations as the hidden state representations.

[0007] In one embodiment, updating the hidden state based on the state matrix output by the selective state space model during each iteration includes: Inputting the target event sequence data into the selective state space model to obtain a first state matrix, a second state matrix, a third state matrix and a time interval; Discretize the first state matrix and the second state matrix based on the time interval to obtain a discretized first state matrix and a discretized second state matrix; Based on the discretized first state matrix and the second state matrix, the current hidden state is updated to obtain an updated hidden state; Based on the third state matrix, the updated hidden state is converted into an output updated hidden state.

[0008] In one embodiment, the target event sequence data is input into the selective state space model to obtain a first state matrix, a second state matrix and a third state matrix; Converting the labeled value of the event type of the target event sequence data into an embedded vector based on a preset embedding matrix, wherein the labeled value represents a one-hot vector of the event type; Obtaining a current first state matrix based on a preset initial state transfer matrix, wherein the first state matrix is ​​a time-invariant matrix; The embedded vector is transformed based on a pre-trained linear projection layer to obtain the second state matrix and the third state matrix, where the second state matrix and the third state matrix are time-varying state matrices.

[0009] In one embodiment, the event sequence data is sequentially input into the intensity function under the Hawkes process for iterative training until a preset convergence condition is met, thereby obtaining a trained event sequence prediction model, including: In each iteration, the event type prediction result and the event time point prediction result are output based on the intensity function under the Hawkes process; Calculating a first loss value based on the first loss function and the event type prediction result; Calculate a second loss value based on the second loss function and the event time point prediction result; Calculating an overall loss value based on the first loss value and the second loss value; When the overall loss value is less than a preset loss threshold, the event sequence prediction model is obtained; otherwise, the parameters of the intensity function under the Hawkes process are updated, and iteration is performed again based on the updated intensity function under the Hawkes process.

[0010] In one embodiment, calculating the overall loss value based on the first loss value and the second loss value includes: Determine the log-likelihood function of the event sequence data input during this iteration; Assigning corresponding weights to the first loss value and the second loss value, and summing them to obtain a loss sum value; The negative number based on the log-likelihood function is added to the loss sum value to obtain the overall loss value.

[0011] In one embodiment, after obtaining the trained event sequence prediction model, the following steps are included: Acquire data to be predicted, and convert the data to be predicted into event sequence data to be predicted; The event sequence data to be predicted is input into the event sequence prediction model, the event type is obtained by prediction through the first linear layer, and the time point is obtained by prediction through the second linear layer.

[0012] In a second aspect, a Hawkes process modeling device based on the Mamba architecture is provided, comprising: A training data set construction unit, used to establish a training data set, wherein the training data set includes a plurality of event sequence data; A hidden state representation generating unit, used for generating a hidden state representation based on the selective state space model and the event sequence data; A modeling unit, used for establishing an intensity function under a Hawkes process based on the hidden state representation; The model training unit is used to input the event sequence data into the intensity function under the Hawkes process in sequence for iterative training until the preset convergence condition is met, and obtain the intensity function under the Hawkes process after training to serve as the event sequence prediction model.

[0013] In a third aspect, a computer device is provided, comprising a memory, a processor, and computer-readable instructions stored in the memory and running on the processor, wherein when the processor executes the computer-readable instructions, the Hawkes process modeling method based on the Mamba architecture as described above is implemented.

[0014] In a fourth aspect, a readable storage medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the Hawkes process modeling method based on the Mamba architecture as described above is implemented.

[0015] The above-mentioned Hawkes process modeling method, device, computer equipment and storage medium based on the Mamba architecture, and its method implementation include: establishing a training data set, the training data set includes multiple event sequence data; generating a hidden state representation based on a selective state space model and the event sequence data; establishing an intensity function under the Hawkes process based on the hidden state representation; inputting the event sequence data into the intensity function under the Hawkes process in sequence for iterative training until the preset convergence conditions are met, and obtaining the intensity function under the trained Hawkes process as an event sequence prediction model. In an embodiment of the present application, a Hawkes process modeling method (MambaHawkes Process, MHP) based on the Mamba architecture is provided, combining a traditional state space model (State Space Model, SSM) and a selective mechanism to construct a selective state space model (Selective SSM). The traditional SSM is good at capturing long-term dependencies, and the selective mechanism enables the model to dynamically update the state representation according to the input event, thereby adapting to the event context. In addition, the selective scanning strategy improves the computational efficiency during training and reasoning. Applying the selective state space model to Hawkes process modeling of complex event sequences provides an efficient and dynamic solution that can flexibly capture long-distance dependencies and accurately model complex dynamic relationships between events. It provides effective methods and tools for the prediction, analysis and management of complex event sequence type data such as medical electronic medical records, social media activity analysis and financial transactions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1It is a flow chart of a Hawkes process modeling method based on the Mamba architecture in one embodiment of the present application; Figure 2 It is a structural schematic diagram of a Hawkes process modeling device based on the Mamba architecture in one embodiment of the present application; Figure 3 It is a schematic diagram of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] In one embodiment, if Figure 1 As shown, a Hawkes process modeling method based on the Mamba architecture is provided, including the following steps: In step S110, a training data set is established, wherein the training data set includes a plurality of event sequence data; In an embodiment of the present application, raw data may be collected, and the raw data may be event sequence data such as financial transaction data, social network behavior data, or medical visit data. It may be collected and acquired through a variety of data sources. Taking medical visit data as an example, it may be connected to the management platforms of multiple hospitals to obtain historical medical visit data of each hospital, or it may be obtained from existing publicly available data sets to obtain historical medical visit data. After the raw data is acquired, the raw data may be cleaned, null values ​​and invalid values ​​may be removed, and the format may be unified. Then, the processed raw data may be integrated into event sequence data, for example, it may be integrated in the order of the time of occurrence. Event sequence data may include event types and time points.

[0020] Exemplarily, the event sequence data S can be expressed as ,in is the timestamp, is the one-hot vector of the event type. It should be noted that the one-hot vector can be obtained by encoding the event type through one-hot encoding.

[0021] It should be noted that the event sequence data can be an asynchronous and irregular event sequence with irregular time intervals and complex inter-event dependency structures. The time intervals of events are non-uniform, which is significantly different from the equally spaced time series. In the event sequence, the time point itself contains important generation process information. Therefore, the modeling method needs to make full use of it to characterize the generation mechanism of the event and reveal the potential laws in the sequence.

[0022] In addition, the dependency structure between events is multi-layered and dynamically changing, not only including self-excitation and self-inhibition effects, but also may involve the mutual dependence between multiple event types in the marking point process, that is, the association between different types of events will affect the probability of future events. In addition, this dependency can be manifested as a short-term association or a long-term association, or both. In addition, these statistical relationships may also be affected by time, external conditions or environmental factors, showing time-varying and situation-dependent characteristics. For example, in the financial market, large transactions may cause short-term market fluctuations and have a lasting impact on the long-term trend of the market; in the medical field, the patient's medical history has a long-term potential impact on future treatment decisions, and this impact will change with the progression of the patient's condition and changes in the treatment plan. Therefore, the present application provides a modeling method that can flexibly adapt to these complex dynamic structures to effectively capture irregularities and diverse statistical dependency characteristics, and better model the generation process behind the event.

[0023] Optionally, after obtaining the event sequence data, it can be divided into a training set, a test set, and a validation set. The data in the training set is used to iteratively train the model, the data in the validation set is used to select the optimal model hyperparameters, and the data in the test set can be used to evaluate the generalization of the optimal model obtained by the validation data.

[0024] In step S120, a hidden state representation is generated based on the selective state space model and the event sequence data; In the embodiments of the present application, the Selective State Space Model (Selective SSM) is constructed by combining the traditional State Space Model (SSM) and the selective mechanism. Among them, the traditional SSM is good at capturing long-term dependencies, and the selective mechanism enables the model to dynamically update the state representation according to the input events, thereby adapting to the event context. In addition, the Selective Scan strategy improves the computational efficiency during training and reasoning. Through the combination of these technologies, MHP shows high efficiency in processing long sequence data and can accurately capture complex dependencies between events.

[0025] In the embodiment of the present application, a recursive algorithm can be used to input each data in each event sequence data into the selective state space model for iterative processing in turn until the hidden state update of each event sequence data is completed, and then the generated state matrix and updated hidden state corresponding to each event sequence data can be obtained. Specifically, an initial hidden state can be initialized, and then the first data in the first event sequence data is input, and a time-invariant state matrix is ​​generated based on the selective state space mechanism. and the time-varying state matrix and and time interval , based on the time interval The state matrix and Converted to a discretized state matrix and , and based on the discretized state matrix and Update the initial hidden state and then pass Output the updated hidden state. The above process can be repeated for each event sequence data to update the last updated hidden state again until all data in all event sequence data are iterated and the updated hidden state corresponding to each data is obtained. Then, all updated hidden states can be spliced ​​into the output of the selective state space according to the output of the selective space state model of different channels under the timestamp, and the output of all selective state spaces can be fitted through the feedforward neural network layer to obtain the hidden state representation.

[0026] In step S130, based on the hidden state representation, an intensity function under the Hawkes process is established; In the embodiment of the present application, the intensity function under the Mamba Hawkes process is constructed, which can be specifically shown as follows: ; in, For the The intensity function of the class event can be defined as follows: ; is the Softplus function, For the moment The hidden state representation of 、 、 are model parameters. The hidden state is represented as generated by the above selective space state model.

[0027] In step S140, the event sequence data is sequentially input into the intensity function under the Hawkes process for iterative training until a preset convergence condition is met, thereby obtaining a trained intensity function under the Hawkes process as an event sequence prediction model.

[0028] In an embodiment of the present application, the event sequence data in the training data set are sequentially input into the intensity function under the Hawkes process constructed above for iterative training, and the event type prediction result and the time point prediction result output by this iteration can be obtained in each iteration process. Based on the pre-constructed loss function, the loss values ​​of the event type prediction result and the time point prediction result are calculated respectively to obtain the total loss value. When the total loss value is greater than / equal to the preset threshold, the parameters of the intensity function under the Hawkes process are updated, and iterated again until the loss value is less than the preset threshold, and the intensity function under the Hawkes process that has been trained is obtained as an event sequence prediction model. Or when the number of iterations reaches the maximum number of iterations, for example 1000 times, the intensity function under the Hawkes process that has been trained can be obtained as an event sequence prediction model.

[0029] In an embodiment of the present application, after the event sequence prediction model is obtained, the model can be evaluated and optimized based on a pre-constructed test set and a validation set to dynamically update the event sequence prediction model to achieve the optimal prediction effect.

[0030] The above-mentioned Hawkes process modeling method based on the Mamba architecture includes: establishing a training data set, the training data set includes multiple event sequence data; generating a hidden state representation based on a selective state space model and the event sequence data; establishing an intensity function under the Hawkes process based on the hidden state representation; inputting the event sequence data into the intensity function under the Hawkes process in sequence for iterative training until the preset convergence conditions are met, and obtaining the intensity function under the trained Hawkes process as an event sequence prediction model. In an embodiment of the present application, a Hawkes process modeling method (Mamba Hawkes Process, MHP) based on the Mamba architecture is provided, which combines the traditional state space model (State Space Model, SSM) and the selective mechanism to construct a selective state space model (SelectiveSSM). The traditional SSM is good at capturing long-term dependencies, and the selective mechanism enables the model to dynamically update the state representation according to the input event, thereby adapting to the event context. In addition, the selective scanning strategy improves the computational efficiency during training and reasoning. Applying the selective state space model to Hawkes process modeling of complex event sequences provides an efficient and dynamic solution that can flexibly capture long-distance dependencies and accurately model complex dynamic relationships between events. It provides effective methods and tools for the prediction, analysis and management of complex event sequence type data such as medical electronic medical records, social media activity analysis and financial transactions.

[0031] In an embodiment of the present application, generating a hidden state representation based on the selective state space model and the event sequence data includes: Inputting each data in each event sequence data into the selective state space model in turn for iteration, and updating the hidden state based on the state matrix output by the selective state space model during each iteration; The hidden states updated in each iteration are concatenated according to the timestamps to obtain multiple output sequences, and all the output sequences are fitted to obtain output state representations as the hidden state representations.

[0032] Optionally, a recursive algorithm can be used to input each data in each event sequence data into the pre-built selective state space model for iterative processing in turn, until all data in all event sequence data are iterated, and then the generated state matrix corresponding to each event sequence data and the updated hidden state can be obtained. Exemplarily, an initial hidden state can be initialized is a zero vector consisting of the first event in the first event sequence data Starting from the beginning, the data are sequentially input into the pre-built selective state space model for processing until the last event in the first event sequence data is The input is processed in the pre-built selective state space model. During each iteration, the iteration can be performed as follows: the unique hot vector of the input event type is converted into an embedding vector, and then a time-invariant state matrix is ​​generated based on the selective state space mechanism. and the time-varying state matrix and and time interval , based on the time interval The state matrix and Converted to a discretized state matrix and , and based on the discretized state matrix and Update the initial hidden state and then pass Output the updated hidden state. The above process can be repeated for each data in the event sequence data to update the last updated hidden state again until all data are iterated and the updated hidden state corresponding to each event sequence data can be obtained. Then, all updated hidden states can be spliced ​​into the output of the selective state space according to the output of the selective space state model of different channels under the timestamp, and the output of all the selective state spaces can be fitted through the feedforward neural network layer to obtain the hidden state representation.

[0033] In one embodiment of the present application, updating the hidden state based on the state matrix output by the selective state space model in each iteration process includes: Inputting the target event sequence data into the selective state space model to obtain a first state matrix, a second state matrix, a third state matrix and a time interval; Discretize the first state matrix and the second state matrix based on the time interval to obtain a discretized first state matrix and a discretized second state matrix; Based on the discretized first state matrix and the second state matrix, the current hidden state is updated to obtain an updated hidden state; Based on the third state matrix, the updated hidden state is converted into an output updated hidden state.

[0034] Among them, the first state matrix is ​​the time-invariant state matrix , the second state matrix is ​​the time-varying state matrix , the third state matrix is ​​the time-varying state matrix . Time interval (set up can be used as initial value).

[0035] Optionally, at each time step , the first state matrix can be fixed or time-invariant, which can be set by the HIPPO theory, and the second state matrix and the third state matrix is the embedding vector from the current input via linear projection Dynamically generated. It should be noted that a trainable linear projection layer can be used , Embedding vector Convert to the corresponding matrix or vector to obtain the second state matrix and the third state matrix. Then, based on the time interval Generate the discretized state matrix and , which can be calculated by the following formula: ; ; in, is the identity matrix and exp is the exponential function.

[0036] Then, the matrix for the current time step can be used and Update hidden state , as shown below: ; Hidden State Through the matrix Converted to output, as follows: ; Continue iterating to the next time step , repeat the above steps until all event sequences are processed.

[0037] Then, the same timestamp can be The updated hidden state output of the selective state space model SSM of different channels under the condition is concatenated into the output of the selected state space ,Will , D represents the number of channels, and the fitting process is performed through the feedforward neural network layer, as follows: ; in, and is the weight matrix, and is the bias vector, is the activation function. Finally, we get , that is, the matrix Row Representative The output state representation at a time point is represented as a hidden state representation as an intensity function of the Hawkes process, which can include All event information before the time point.

[0038] In one embodiment of the present application, the target event sequence data is input into the selective state space model to obtain a first state matrix, a second state matrix and a third state matrix; Converting the labeled value of the event type of the target event sequence data into an embedded vector based on a preset embedding matrix, wherein the labeled value represents a one-hot vector of the event type; Obtaining a current first state matrix based on a preset initial state transfer matrix, wherein the first state matrix is ​​a time-invariant matrix; The embedded vector is transformed based on a pre-trained linear projection layer to obtain the second state matrix and the third state matrix, where the second state matrix and the third state matrix are time-varying state matrices.

[0039] Optionally, for the input target event sequence data, you can first embed the matrix The event type's value Convert to embedding vector , the formula is: ; in, The dimension is The embedding matrix of is the dimension of the embedding vector, is the total number of event types.

[0040] At initialization, the first state matrix The initial value of can be set according to the HIPPO theory, using or To help the model acquire the ability to capture long-distance dependencies.

[0041] For the second state matrix and the third state matrix Then the pre-trained linear projection layer , Embedding vector The conversion is as follows: ; in, Represents the time interval between two consecutive events.

[0042] The dimensions of a matrix or vector can be as follows: , is the state transfer matrix, describing the change of hidden state.

[0043] , is the input matrix, which projects the input into the state space.

[0044] , is the output matrix, which projects the hidden state into the output space.

[0045] in, is the dimension of the hidden state, , Parameters obtained for the pre-trained linear projection layer.

[0046] In one embodiment of the present application, the event sequence data is sequentially input into the intensity function under the Hawkes process for iterative training until a preset convergence condition is met, thereby obtaining a trained intensity function under the Hawkes process, including: In each iteration, the event type prediction result and the event time point prediction result are output based on the intensity function under the Hawkes process; Calculating a first loss value based on the first loss function and the event type prediction result; Calculate a second loss value based on the second loss function and the event time point prediction result; Calculating an overall loss value based on the first loss value and the second loss value; When the overall loss value is less than a preset loss threshold, the event sequence prediction model is obtained; otherwise, the parameters of the intensity function under the Hawkes process are updated, and the intensity function under the updated Hawkes process is iterated again until the preset convergence condition is met, thereby obtaining the intensity function under the trained Hawkes process.

[0047] Optionally, given the current hidden state based on the selective spatial state model , we can predict the type of the next event through a linear layer: ); in, Softmax The function is used to calculate the probability distribution of each event type. The function is used to determine the independent variable corresponding to the maximum value of a given function. is the linear transformation matrix. Similarly, based on the selective spatial state model, the current hidden state is given , we can predict the time point of the next event through another linear layer: ; in, is the linear transformation matrix.

[0048] Based on the type of the predicted event, the first loss function may be a cross entropy loss function, and the first loss value may be specifically calculated by the following formula: ; in, is the activation function, is the linear transformation matrix, Indicates the type of event being predicted, means time.

[0049] In addition, the second loss function can be a mean square error (MSE) loss, which is shown as follows: ; in, The time point that can be predicted by the model, It refers to the time point predicted by the previous event model. means time, means time.

[0050] Based on the first loss value calculated by the above-mentioned first loss function and the second loss value calculated by the second loss function, the overall loss value can be calculated. When the overall loss value is less than the preset loss threshold, the event sequence prediction model is obtained. Otherwise, the parameters of the intensity function under the Hawkes process are updated, and the intensity function under the updated Hawkes process is iterated again until the preset convergence condition is met. The intensity function under the trained Hawkes process is obtained as an event sequence prediction model for predicting the actual event sequence to be predicted. It can be understood that the first loss function and the second loss function can also use other loss functions to realize the calculation of loss values, and this application does not limit this.

[0051] In an embodiment of the present application, calculating the overall loss value based on the first loss value and the second loss value includes: Determine the log-likelihood function of the event sequence data input during this iteration; Assigning corresponding weights to the first loss value and the second loss value, and summing them to obtain a loss sum value; The negative number based on the log-likelihood function is added to the loss sum value to obtain the overall loss value.

[0052] Optionally, the log-likelihood function can be calculated by the following formula: ; in, Indicates integration over time. express The conditional intensity function at time, express The conditional intensity function at time.

[0053] Therefore, the overall loss value can be calculated by the following formula: ; in, is the log-likelihood function of the event sequence, , is a hyperparameter that controls the weights of different loss terms.

[0054] Based on the overall loss value obtained in each iteration, determine whether it is less than the preset loss threshold. If not, adjust the hyperparameters and input the event sequence data again for the next iteration until the overall loss value is less than the preset loss threshold, and the training ends.

[0055] In one embodiment of the present application, after obtaining the trained event sequence prediction model, the following steps are included: Acquire data to be predicted, and convert the data to be predicted into event sequence data to be predicted; The event sequence data to be predicted is input into the event sequence prediction model, the event type is obtained by prediction through the first linear layer, and the time point is obtained by prediction through the second linear layer.

[0056] The data to be predicted can be obtained, converted into event sequence data to be predicted, and input into the trained event sequence prediction model for prediction processing. The event sequence prediction model may include a first linear layer and a second linear layer. The event type is predicted based on the first linear layer, and the time point is predicted through the second linear layer. Therefore, the time point prediction result and the event type prediction result can be obtained. Data analysis and management can be performed based on the prediction results.

[0057] In the embodiment of the present application, in order to further verify the effect of the MHP provided in the present application, experiments can be conducted on different data sets (such as financial transactions, StackOverflow, MIMIC-II, etc.) to verify the significant advantages of MHP in event prediction and fitting accuracy. This scheme is evaluated and compared by log-likelihood, accuracy, and root mean square error (RMSE). Among them, the log-likelihood is used to measure the degree of fit of the model to the event sequence. The higher the value, the better the fitting effect; the accuracy measures the classification performance of the model in event type prediction. The higher the accuracy, the better the performance of the model in capturing the complex dependencies between events; the root mean square error is used to evaluate the accuracy of the model in predicting the time point of event occurrence. The lower the root mean square error, the more accurately the model can predict the time of event occurrence. Through the comprehensive analysis of these three indicators, the advantages of Mamba Hawkes Process and traditional neural network baseline models in fitting accuracy, event classification and time prediction can be comprehensively compared, thereby verifying the effectiveness and superiority of the patent method.

[0058] The experimental results of log-likelihood, accuracy and root mean square error (RMSE) of different models on different data sets are shown in Tables 1 to 3: Table 1: Log-Likelihood table of different models on different data sets

[0059] Table 2: Accuracy of different models on different data sets

[0060] Table 3: Root mean square error (RMSE) table of different models on different data sets

[0061] Among them, RMTTP is a cyclic marking time point process, which uses a recurrent neural network RNN ​​architecture to predict the time of subsequent events. NHP represents a neural Hawkes process, which combines a neural network with a Hawkes process to predict accuracy. SAHP represents the combination of a self-attention neural network with a Hawkes process to predict accuracy, THP represents the application of the Transformer structure to the Hawkes process in the Transformer Hawkes process, and introduces absolute position encoding in the model construction, and MHP represents the modeling method provided in this application. The MHP method can not only be used alone, but also can be used as a technical module in conjunction with other technical methods. Here, MHP-E means that MHP is used as a technical module and THP together to form a combined model. Among them, RMTTP, NHP, SAHP and THP are all currently known models. The Financial, Synthetic, SO, Retweet, and Mimic-II respectively represent five different public data sets commonly used in the literature on neural point process modeling, that is, Financial represents a financial data set. For example, the data set contains stock trading records within a trading day. The sequences in the data set are longer, and the events are divided into types: "buy" and "sell". The average length of the dataset is 2074. Synthetic represents a synthetic dataset. The SO dataset collects data from the question-and-answer website StackOverflow, which treats user interaction history as a time series. The average length of the sequence in this dataset is 72, the minimum is 4, the maximum is 736, and it has 22 event types. The Retweet dataset compiles various push chains. Each chain includes an original tweet posted by a user, and subsequent reply tweets. The accompanying information includes the time of each tweet and the user's identifier. Its average length is 109, the minimum is 50, and the maximum is 264. Event types are divided into 3 types according to the number of followers: "small, medium, and large." Mimic-II represents a medical dataset, which includes multiple patients' visit data, each patient's visit is regarded as a different sequence, and each event is marked with a timestamp and diagnosis.

[0062] It can be seen from Tables 1 to 3 above that the log-likelihoods of the MHP model provided in this application on different data sets are 0.974, 0.391, 0.993, 0.180 and 0.996, respectively, while the log-likelihoods of MHP-E on different data sets are 0.966, 0.407, 0.956, 2.016 and 1.186, respectively, which are significantly higher than the log-likelihoods of the models RMTPP, NHP, SAHP and THP. Therefore, its fitting effect is significantly better than other models. The accuracy of the MHP model provided in this application on different data sets is 62.5, 85.5 and 45.4, respectively, while the accuracy of the MHP-E model on different data sets is 62.7, 85.5 and 46.5, respectively, which is obviously higher than the models RMTPP, NHP, SAHP and THP. Therefore, the model provided in this application performs better in capturing the complex dependencies between events. The root mean square error (RMSE) of the MHP model provided in this application on different data sets is 0.592, 0.687 and 1.429, respectively, while the root mean square error (RMSE) of the MHP-E model on different data sets is 0.556, 0.588 and 1.372, respectively, which is significantly lower than the models RMTPP, NHP, SAHP and THP. It can be seen that the model provided in this application can more accurately predict the time of event occurrence.

[0063] In an embodiment of the present application, a Hawkes process modeling method (MambaHawkes Process, MHP) based on the Mamba architecture is provided, which combines the traditional state space model (State Space Model, SSM) and the selective mechanism to construct a selective state space model (Selective SSM). The traditional SSM is good at capturing long-term dependencies, and the selective mechanism enables the model to dynamically update the state representation according to the input event, thereby adapting to the event context. In addition, the selective scanning (Selective Scan) strategy improves the computational efficiency during training and reasoning. Applying the selective state space model to the Hawkes process modeling of complex event sequences provides an efficient and dynamic solution that can flexibly capture long-distance dependencies and accurately model complex dynamic relationships between events, providing effective methods and tools for the prediction, analysis and management of complex event sequence type data such as medical electronic medical records, social media activity analysis, and financial transactions.

[0064] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0065] In one embodiment, a Hawkes process modeling device based on the Mamba architecture is provided, and the Hawkes process modeling device based on the Mamba architecture corresponds one-to-one to the Hawkes process modeling method based on the Mamba architecture in the above embodiment. Figure 2 As shown, the Hawkes process modeling device based on the Mamba architecture includes a training data set construction unit 10, a hidden state representation generation unit 20, a modeling unit 30 and a model training unit 40. The functional modules are described in detail as follows: A training data set construction unit 10 is used to establish a training data set, wherein the training data set includes a plurality of event sequence data; A hidden state representation generating unit 20, configured to generate a hidden state representation based on the selective state space model and the event sequence data; A modeling unit 30, for establishing an intensity function under a Hawkes process based on the hidden state representation; The model training unit 40 is used to sequentially input the event sequence data into the intensity function under the Hawkes process for iterative training until a preset convergence condition is met, thereby obtaining a trained event sequence prediction model.

[0066] In one embodiment of the present application, the hidden state representation generating unit 20 is further used for: Inputting each data in each event sequence data into the selective state space model in turn for iteration, and updating the hidden state based on the state matrix output by the selective state space model during each iteration; The hidden states updated in each iteration are concatenated according to the timestamps to obtain multiple output sequences, and all the output sequences are fitted to obtain output state representations as the hidden state representations.

[0067] In one embodiment of the present application, the hidden state representation generating unit 20 is further used for: Inputting the target event sequence data into the selective state space model to obtain a first state matrix, a second state matrix, a third state matrix and a time interval; Discretize the first state matrix and the second state matrix based on the time interval to obtain a discretized first state matrix and a discretized second state matrix; Based on the discretized first state matrix and the second state matrix, the current hidden state is updated to obtain an updated hidden state; Based on the third state matrix, the updated hidden state is converted into an output updated hidden state.

[0068] In one embodiment of the present application, the hidden state representation generating unit 20 is further used to: convert the label value of the event type of the target event sequence data into an embedded vector based on a preset embedding matrix, wherein the label value represents a unique hot vector of the event type; Obtaining a current first state matrix based on a preset initial state transfer matrix, wherein the first state matrix is ​​a time-invariant matrix; The embedded vector is transformed based on a pre-trained linear projection layer to obtain the second state matrix and the third state matrix, where the second state matrix and the third state matrix are time-varying state matrices.

[0069] In one embodiment of the present application, the model training unit 40 is further used to: In each iteration, the event type prediction result and the event time point prediction result are output based on the intensity function under the Hawkes process; Calculating a first loss value based on the first loss function and the event type prediction result; Calculate a second loss value based on the second loss function and the event time point prediction result; Calculating an overall loss value based on the first loss value and the second loss value; When the overall loss value is less than a preset loss threshold, the event sequence prediction model is obtained; otherwise, the parameters of the intensity function under the Hawkes process are updated, and iteration is performed again based on the updated intensity function under the Hawkes process.

[0070] In one embodiment of the present application, the model training unit 40 is further used to: Determine the log-likelihood function of the event sequence data input during this iteration; Assigning corresponding weights to the first loss value and the second loss value, and summing them to obtain a loss sum value; The negative number based on the log-likelihood function is added to the loss sum value to obtain the overall loss value.

[0071] In an embodiment of the present application, the device further includes an actual prediction unit 40, which is used to: Acquire data to be predicted, and convert the data to be predicted into event sequence data to be predicted; The event sequence data to be predicted is input into the event sequence prediction model, the event type is obtained by prediction through the first linear layer, and the time point is obtained by prediction through the second linear layer.

[0072] In an embodiment of the present application, a Hawkes process modeling method (MambaHawkes Process, MHP) based on the Mamba architecture is provided, which combines the traditional state space model (State Space Model, SSM) and the selective mechanism to construct a selective state space model (Selective SSM). The traditional SSM is good at capturing long-term dependencies, and the selective mechanism enables the model to dynamically update the state representation according to the input event, thereby adapting to the event context. In addition, the selective scanning (Selective Scan) strategy improves the computational efficiency during training and reasoning. Applying the selective state space model to the Hawkes process modeling of complex event sequences provides an efficient and dynamic solution that can flexibly capture long-distance dependencies and accurately model complex dynamic relationships between events, providing effective methods and tools for the prediction, analysis and management of complex event sequence type data such as medical electronic medical records, social media activity analysis, and financial transactions.

[0073] For the specific definition of a Hawkes process modeling device based on the Mamba architecture, please refer to the definition of a Hawkes process modeling method based on the Mamba architecture above, which will not be repeated here. Each module in the above-mentioned Hawkes process modeling device based on the Mamba architecture can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0074] In one embodiment, a computer device is provided. The computer device may be a terminal device, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer-readable instructions. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a Hawkes process modeling method based on the Mamba architecture is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0075] In an embodiment of the present application, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of a Hawkes process modeling method based on the Mamba architecture as described above are implemented.

[0076] In an embodiment of the application, a readable storage medium is provided, wherein the readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of a Hawkes process modeling method based on the Mamba architecture as described above are implemented.

[0077] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0078] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0079] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A Hawkes process modeling method based on Mamba architecture, characterized in that: The method comprises: Establishing a training data set, wherein the training data set includes a plurality of event sequence data; generating a hidden state representation based on the selective state space model and the event sequence data; Based on the hidden state representation, an intensity function under the Hawkes process is established; The event sequence data are sequentially input into the intensity function under the Hawkes process for iterative training until a preset convergence condition is met, thereby obtaining a trained intensity function under the Hawkes process as an event sequence prediction model.

2. The Hawkes process modeling method based on Mamba architecture as claimed in claim 1, characterized in that: The generating a hidden state representation based on the selective state space model and the event sequence data comprises: Inputting each data in each event sequence data into the selective state space model in turn for iteration, and updating the hidden state based on the state matrix output by the selective state space model during each iteration; The hidden states updated in each iteration are concatenated according to the timestamps to obtain multiple output sequences, and all the output sequences are fitted to obtain output state representations as the hidden state representations.

3. The Hawkes process modeling method based on Mamba architecture as claimed in claim 2, characterized in that: The updating of the hidden state based on the state matrix output by the selective state space model in each iteration process comprises: Inputting the target event sequence data into the selective state space model to obtain a first state matrix, a second state matrix, a third state matrix and a time interval; Discretize the first state matrix and the second state matrix based on the time interval to obtain a discretized first state matrix and a discretized second state matrix; Based on the discretized first state matrix and the second state matrix, the current hidden state is updated to obtain an updated hidden state; Based on the third state matrix, the updated hidden state is converted into an output updated hidden state.

4. The Hawkes process modeling method based on Mamba architecture as claimed in claim 3, characterized in that: The target event sequence data is input into the selective state space model to obtain a first state matrix, a second state matrix and a third state matrix; Converting the labeled value of the event type of the target event sequence data into an embedded vector based on a preset embedding matrix, wherein the labeled value represents a one-hot vector of the event type; Obtaining a current first state matrix based on a preset initial state transfer matrix, wherein the first state matrix is ​​a time-invariant matrix; The embedded vector is transformed based on a pre-trained linear projection layer to obtain the second state matrix and the third state matrix, where the second state matrix and the third state matrix are time-varying state matrices.

5. The Hawkes process modeling method based on Mamba architecture as claimed in claim 1, characterized in that: The step of sequentially inputting the event sequence data into the intensity function under the Hawkes process for iterative training until a preset convergence condition is met, thereby obtaining a trained intensity function under the Hawkes process, comprises: In each iteration, the event type prediction result and the event time point prediction result are output based on the intensity function under the Hawkes process; Calculate a first loss value based on the first loss function and the event type prediction result; Calculate a second loss value based on the second loss function and the event time point prediction result; Calculating an overall loss value based on the first loss value and the second loss value; When the overall loss value is less than a preset loss threshold, the event sequence prediction model is obtained; otherwise, the parameters of the intensity function under the Hawkes process are updated, and iteration is performed again based on the updated intensity function under the Hawkes process.

6. The Hawkes process modeling method based on Mamba architecture as claimed in claim 5, characterized in that: The calculating the overall loss value based on the first loss value and the second loss value includes: Determine the log-likelihood function of the event sequence data input during this iteration; Assigning corresponding weights to the first loss value and the second loss value, and summing them to obtain a loss sum value; The negative number based on the log-likelihood function is added to the loss sum value to obtain the overall loss value.

7. The Hawkes process modeling method based on Mamba architecture as claimed in claim 1, characterized in that: After obtaining the trained event sequence prediction model, the following steps are included: Acquire data to be predicted, and convert the data to be predicted into event sequence data to be predicted; The event sequence data to be predicted is input into the event sequence prediction model, the event type is obtained by prediction through the first linear layer, and the time point is obtained by prediction through the second linear layer.

8. A Hawkes process modeling device based on Mamba architecture, characterized in that: The device comprises: A training data set construction unit, used to establish a training data set, wherein the training data set includes a plurality of event sequence data; A hidden state representation generating unit, used for generating a hidden state representation based on the selective state space model and the event sequence data; A modeling unit, used for establishing an intensity function under a Hawkes process based on the hidden state representation; The model training unit is used to input the event sequence data into the intensity function under the Hawkes process in sequence for iterative training until the preset convergence condition is met, and obtain the intensity function under the Hawkes process after training to serve as the event sequence prediction model.

9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executed on the processor, characterized in that: When the processor executes the computer-readable instructions, the Hawkes process modeling method based on the Mamba architecture as described in any one of claims 1 to 7 is implemented.

10. A readable storage medium having computer readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, the Hawkes process modeling method based on the Mamba architecture as claimed in any one of claims 1 to 7 is implemented.

Images