Critical event prediction method based on improved neuro-hox process
By improving the neural Hawkes process and combining feature engineering with multi-layer perceptrons, we have solved the problem of predicting abnormal events in complex event sequences, achieved accurate predictions of the probability and intensity levels of future abnormal events, and improved the accuracy and reliability of the predictions.
Patent Information
- Application Number
- CN202510823136.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-19
- Publication Date
- 2025-10-03
AI Technical Summary
Existing technologies have difficulty in effectively predicting the probability and intensity level of abnormal events in complex event sequences, especially in the case of multiple sources and multiple event cascades, and there is a lack of efficient prediction methods.
An improved neural Hawkes process is adopted to conduct multi-task learning through feature engineering, conditional intensity function, public opinion field paradigm and representation learning, combined with Transformer mechanism and multi-layer perceptron to predict the probability and intensity level of future abnormal events.
It achieves high-precision prediction of abnormal events, accurately reflects various influencing factors in the information dissemination process, obtains high-quality node representation, and improves the accuracy and reliability of prediction.
Smart Images
Figure CN120745908A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information technology, and in particular to a crisis event prediction method based on an improved neural Hawkes process. Background Art
[0002] Real-life events form complex sequences and may originate from multiple sources. For example, interstate relations involve multiple events, each stemming from factors such as economic interests and territorial disputes. Each source can also generate multiple sequences, forming a cascade of events. Furthermore, weekly events have distinct intensity levels, ranging from peaceful to various abnormal states. Given this complexity, a method is needed that can predict the probability and intensity of abnormal events for the next week by inputting historical data. This is of great significance to fields such as public opinion monitoring and information dissemination. Summary of the Invention
[0003] In order to predict the probability and intensity level of abnormal events, the present invention proposes a crisis event prediction method based on an improved neural Hawkes process.
[0004] The technical solution adopted in the present invention is:
[0005] A crisis event prediction method based on an improved neural Hawkes process, the method comprising:
[0006] Step 101: Perform feature engineering on the original event sequence;
[0007] Step 102: Use the Hawkes process to describe the conditional intensity during the event propagation process and construct a conditional intensity function;
[0008] Step 103: Construct the public opinion field paradigm based on the conditional intensity function;
[0009] Step 104: Use the public opinion field paradigm to learn the public opinion field representation and obtain the embedding of the event node;
[0010] Step 105: Based on the event node embedding, the Transformer mechanism and multi-layer perceptron are used to perform two tasks: predicting the probability of abnormal events occurring in the future and the event intensity level;
[0011] Step 106: Weight the loss functions of the two tasks and perform multi-task learning.
[0012] Furthermore, the specific process of step 101 is as follows:
[0013] First, the original event sequence is cleaned to remove irrelevant characters, punctuation marks, and stop words. Next, text encoding is performed, and the Sentence-BERT method is used to convert the event text summary into a numerical vector. Then, label encoding is performed, and the event intensity level label is converted into a numerical form using one-hot encoding. Finally, label smoothing is performed. Among them, the event intensity level is divided into multiple levels, that is, multiple labels. A degree of abnormality is defined for each label, and the probability prediction is converted into a regression prediction problem.
[0014] Furthermore, the specific process of step 102 is as follows:
[0015] The Hawkes process is used to describe the conditional intensity of the event propagation process, and the event m k The conditional intensity function of the i-th cascade sequence at time t for:
[0016]
[0017] In the formula, θ represents the parameter set, w h , w Φ and w b They represent the learnable parameters, Represents event m k Features, It is used to measure the impact of the forwarding event that occurred earlier in the sequence on the strength of the condition at the future moment, h(t j ) represents t j The hidden state variable at the moment, g(tt j ) represents the end of time t j The impact of the forwarding behavior on the condition strength at time t, It is used to measure the impact of other forwarding sequences of the same event on this forwarding sequence. The expression of Φ(t) is as follows:
[0018]
[0019] Among them, R k Represents event m k Number of sequences sent, t c Represents the cth cascade sequence The last forwarding moment, Indicates the basic strength of the cascade sequence, with event m k The initial characteristics To measure.
[0020] Furthermore, the specific process of step 103 is as follows:
[0021] Each event node is defined as the center of the public opinion field, ending at the observation time t oThe propagation path of the event is defined as the public opinion field centered on the event node; the conditional intensity functions of all propagation paths of each event are summed to represent the public opinion field energy of the event; the event node m is used as the center of the public opinion field. k The public opinion field formed as the center k Its energy is defined as:
[0022]
[0023] Where, t i is the last forwarding time in the i-th cascade sequence.
[0024] Furthermore, the specific process of step 104 is as follows:
[0025] (1) Normalize the energy of all public opinion fields: E(P k )←E(P k ) / ∑E(P k ), and then use the graph attention mechanism to update the event node, that is, the representation of the central node of the public opinion field;
[0026] (2) For friendship relationships between users, we use the graph attention mechanism to aggregate events between nodes; for forwarding relationships between users, we use the graph attention mechanism to aggregate events between nodes; for interest relationships between users and events, we combine the public opinion field energy and the graph attention mechanism to aggregate features between users and events;
[0027] (3) All the multiple representations of the user are concatenated and then transformed to obtain the final node representation of the user;
[0028] (4) Repeat steps (1) to (3) twice to obtain the final node embedding of the event node.
[0029] Furthermore, the specific process of step 105 is as follows:
[0030] In order to predict the level of events in the next week, we first average the embedding vectors of multiple events in each week to obtain a weekly embedding vector of size d for that week, and then concatenate the embedding vectors of the four weeks together to form an input matrix H∈R b×4×d ; Where b represents the batch size fed into the model;
[0031] A time coding matrix T is introduced to encode the sequence of events. Specifically, the start time of each week is first converted to a Unix timestamp, then normalized, and finally mapped to the same dimension as the text features through a multi-layer perceptron (MLP) to obtain the time coding matrix T.
[0032] Use H and T to implement the self-attention mechanism in Transformer:
[0033] Q=W Q (H+f t (T))
[0034] K=W K (H+f t (T))
[0035] V=W V (H+f t (T))
[0036]
[0037] Among them, W Q 、W K and W V Both represent weight conversion matrices, f t is a parameter matrix used to convert the timestamp matrix to the same size as the embedding matrix;
[0038] After multi-layer Transformer processing, the event embedding matrix H′ for four weeks is obtained. The last time step, that is, the output of the current week, is used as the representation of the sequence to capture the latest state of the sequence.
[0039] After obtaining the overall representation, we first use a multi-layer perceptron (MLP) to predict the probability of abnormal events occurring in the next week:
[0040]
[0041] in, Represents the predicted probability value of abnormal events occurring in the next week, and then calculates the MSE loss function with the actual probability: Where C1 and Represent the true probability and predicted probability of all samples in the training set respectively;
[0042] Then, a multi-layer perceptron (MLP) is used to perform category prediction and obtain the prediction results of the event intensity level:
[0043]
[0044] in, It is the probability distribution of intensity level of a single sample with a length of 5. The category represented by the position with the largest probability is the predicted intensity category; 0, 1, 2, 3, and 4 represent the five event intensity levels respectively; finally, the intensity is calculated using the cross entropy loss function Among them, C2 and are the true categories and predicted categories of all samples in the training set, respectively.
[0045] Furthermore, the specific process of step 106 is as follows:
[0046] Add the losses of the two tasks to perform multi-task learning:
[0047] L=μL1+ρL2
[0048] Among them, μ and ρ represent the weights of the anomaly probability prediction loss function and the intensity level loss function, respectively.
[0049] The advantages of the present invention over the prior art are:
[0050] 1. This paper proposes using the Hawkes process to describe the strength of conditions during information dissemination, and uses this strength to describe the ability of information to attract users to forward it, namely, the energy of the public opinion field. Conditional strength and public opinion field energy can accurately reflect the various influencing factors in the information dissemination process from multiple perspectives.
[0051] 2. This invention links public opinion fields with representation learning, effectively extracting and integrating multiple relationships in the information dissemination process to obtain high-quality node representations. This high-quality node representation can achieve more accurate event prediction results. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 A schematic diagram of cascade prediction formed by an information node in an embodiment of the present invention.
[0053] Figure 2 This is a flowchart of a crisis event prediction method based on an improved neural Hawkes process proposed in an embodiment of the present invention. DETAILED DESCRIPTION
[0054] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below with reference to the accompanying drawings.
[0055] A crisis event prediction method based on improved neural Hawkes process, such as Figure 2 As shown, the method includes the following steps:
[0056] Step 101: Feature engineering is performed on the original event sequence. Specifically, the text is cleaned to remove irrelevant characters, punctuation, and stop words. Next, text encoding is performed, using the Sentence-BERT method to convert the event text summary into a numerical vector. This method, based on an extension of the pre-trained BERT model, uses a twin and three-level network structure to obtain semantically meaningful sentence embeddings. Label encoding is then performed, using one-hot encoding to convert event intensity labels such as "stable," "fluctuating," "critical," "confrontational," and "rupture" into numerical form. To more accurately predict the probability of an abnormal event occurring in the next week, an abnormality level is assigned to each label, and the probability prediction is converted into a regression prediction problem. Specifically, the abnormality level of the "stable" label is defined as 0, and the abnormality levels of "fluctuating," "critical," "confrontational," and "rupture" are defined as 0.4, 0.6, 0.8, and 1.0, respectively. A high value such as 0.7 or 0.8 for the predicted intensity of the next week indicates a high probability of an abnormal event. Finally, label smoothing is performed to slightly adjust the absolute one-hot labels to reduce overfitting and alleviate class imbalance.
[0057] Step 102: Use the Hawkes process to describe the conditional intensity during the event propagation process and construct a conditional intensity function;
[0058] The specific steps are as follows:
[0059] make Indicates information m k Observe at time t o The previous cascade propagation sequence i, user u j In t j Forwarded the message m k ;
[0060] Inspired by the Hawkes process, its classic conditional intensity function is where μ is the basic intensity, which represents the basic probability of an event occurring without any influence of previous events; ψ(tt j ) is a pre-specified decay function used to describe the decay of the influence of past events on future events, that is, as the time interval tt j As t increases, the influence of past events on the probability of current events gradually decreases. We further introduce the neural Hawkes process, whose expression is λ(t)=f(w T h(t)), where is a softplus function that ensures that the strength is positive, where β is a parameter. Here h(t) is the hidden state of the historical event encoded by the continuous time LSTM, and w is the learnable weight. The Hawkes process is used to describe the conditional strength of the information propagation process. The information mk The conditional intensity function of the i-th cascade sequence at time t for:
[0061]
[0062] In the formula, θ represents the parameter set, w h , w Φ and w b Both represent learnable parameters, Indicates information m k The first term on the right side of the equation It is used to measure the impact of the forwarding event that occurred earlier in the sequence on the strength of the condition at the future moment; h(t j ) represents t j The hidden state variable at the moment, the present invention extracts the hidden state variable at the moment based on GRU; g(tt j ) represents the end of time t j The impact of the forwarding behavior on the conditional strength at time t. Generally speaking, the impact of past forwarding events on future forwarding events is not fixed, but will weaken over time; therefore, g(tt j ) is defined as follows: It can be found that with t j The increase of g(tt j ) increases exponentially, indicating that the closer the forwarding behavior occurs to time t, the greater its impact;
[0063] The second term on the right side of the equation Used to measure the impact of other forwarding sequences of the same message on this forwarding sequence; Figure 1 As shown in Figure 2, multiple cascade sequences sent by an information source are all based on the same information source, so there is a certain connection between them. The expression of Φ(t) is as follows:
[0064]
[0065] Among them, R k Indicates information m k Number of sequences sent, t c Represents the cth cascade sequence The last forwarding moment in the equation; the third term on the right side of the equation Indicates the basic strength of the cascade sequence, using information m k The initial characteristics To measure.
[0066] 103: Based on the definition in step 102, the conditional intensity function of each event is obtained to construct the public opinion field paradigm. The specific process is as follows:
[0067] Each event node is defined as the center of the public opinion field, ending at the observation time t o The propagation path of the event is defined as the public opinion field centered on the event node;
[0068] Next, the present invention calculates all paths of each information source at the observation time t o The sum of the conditional strength function at each location represents the energy of the public opinion field where the information source is located. This is because the conditional strength describes the possibility of a forwarding event in the future. The greater the conditional strength, the greater the ability to attract users to forward, and therefore the greater the energy of the public opinion field. Specifically, taking the information node m as an example, k The public opinion field formed as the center k Its energy is defined as:
[0069]
[0070] Among them, t i is the last forwarding time in the i-th cascade sequence.
[0071] 104: Based on the public opinion field paradigm defined in step 103, public opinion field representation learning is performed to obtain the embedding of event nodes. The specific process is as follows:
[0072] (1) First, normalize the energy of all public opinion fields: E(P k )←E(P k ) / ∑E(P k ), and then use the graph attention mechanism to update the event node, that is, the representation of the central node of the public opinion field; specifically:
[0073]
[0074] Among them, N k It is m k The user neighbor set, α k,i It is m k The attention score between neighbors is expressed as:
[0075]
[0076] Here h k Represents event node m k Embedding, h i Represents event node m k User neighbors v i Embedding of. It means g(t0-t i ) is used to describe the importance of forwarding behavior at different times, att(h k ,h i ,Φ)=σ(Φ T ·(h k||h i )) is the calculation formula of the attention score, is the LeakyReLU activation function, Φ is the weight of the attention mechanism, and || represents the splicing operation;
[0077] (2) For friendship relationships between users, we use the graph attention mechanism to aggregate information between nodes; for forwarding relationships between users, we use the graph attention mechanism to aggregate information between nodes; for the interest relationship between users and information, we consider that a user may be involved in forwarding multiple pieces of information at the same time, and different information has different energy in the public opinion field. Therefore, we combine the public opinion field energy and the graph attention mechanism to aggregate features between users and information.
[0078] (3) All the multiple representations of the user are concatenated and then transformed to obtain the final node representation of the user;
[0079] (4) Repeat steps (1) to (3) twice to obtain the final node embedding of the event node.
[0080] 105: Based on event node embedding, use the Transformer mechanism and multi-layer perceptron to predict the probability and intensity level of future abnormal events; the details are as follows:
[0081] In order to predict the level of events in the next week, we first average the embedding vectors of multiple events in each week to obtain a weekly embedding vector of size d for that week, and then concatenate the embedding vectors of the four weeks together to form an input matrix H∈R b×4×d ; where b represents the batch size fed into the model.
[0082] A time coding matrix T is introduced to encode the sequence of events. Specifically, the start time of each week is first converted to a Unix timestamp, then normalized, and finally mapped to the same dimension as the text features through a multi-layer perceptron (MLP) to obtain the time coding matrix T.
[0083] Use H and T to implement the self-attention mechanism in Transformer:
[0084] Q=W Q (H+f t (T))
[0085] K=W K (H+f t (T))
[0086] V=W V (H+f t (T))
[0087]
[0088] Where W Q 、W K and W V Both represent weight conversion matrices, f t is a parameter matrix used to convert the timestamp matrix to the same size as the embedding matrix.
[0089] After multi-layer Transformer processing, the event embedding matrix H′ for four weeks is obtained. The last time step, that is, the output of the current week, is used as the representation of the sequence to capture the latest state of the sequence.
[0090] After obtaining the overall representation, we first use a multi-layer perceptron (MLP) to predict the probability of abnormal events occurring in the next week:
[0091]
[0092] in, Represents the predicted probability value of abnormal events occurring in the next week, and then calculates the MSE loss function with the actual probability: Where C1 and Represent the true probability and predicted probability of all samples in the training set respectively;
[0093] Then, a multi-layer perceptron (MLP) is used to perform category prediction and obtain the prediction results of the event intensity level:
[0094]
[0095] in, is the probability distribution of intensity levels for a single sample length of 5. The category represented by the position with the largest probability is the predicted intensity category; 0, 1, 2, 3, and 4 represent "peace", "tension", "crisis", "conflict", and "war" respectively; finally, the cross entropy loss function is used to calculate the intensity Among them, C2 and are the true categories and predicted categories of all samples in the training set, respectively.
[0096] 106: Multi-task learning is performed by weighted addition of anomaly probability and intensity level loss functions. Specifically, in order to optimize two tasks at the same time, the losses of the two tasks are added together for multi-task learning:
[0097] L=μL1+ρL2
[0098] Among them, μ and ρ represent the weights of the anomaly probability prediction loss function and the intensity level loss function, respectively.
[0099] Unless otherwise specified, the embodiments of the present invention do not limit the models of the components. Any component that can perform the above functions may be used.
[0100] Those skilled in the art will understand that the accompanying drawings are only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A crisis event prediction method based on an improved neural Hawkes process, characterized in that: The method comprises: Step 101: Perform feature engineering on the original event sequence; Step 102: Use the Hawkes process to describe the conditional intensity during the event propagation process and construct a conditional intensity function; Step 103: Construct the public opinion field paradigm based on the conditional intensity function; Step 104: Use the public opinion field paradigm to learn the public opinion field representation and obtain the embedding of the event node; Step 105: Based on the event node embedding, the Transformer mechanism and multi-layer perceptron are used to perform two tasks: predicting the probability of abnormal events occurring in the future and the event intensity level; Step 106: Weight the loss functions of the two tasks and perform multi-task learning.
2. The crisis event prediction method based on the improved neural Hawkes process according to claim 1, characterized in that: The specific process of step 101 is as follows: First, the original event sequence is cleaned to remove irrelevant characters, punctuation marks, and stop words. Next, text encoding is performed, and the Sentence-BERT method is used to convert the event text summary into a numerical vector. Then, label encoding is performed, and the event intensity level label is converted into a numerical form using one-hot encoding. Finally, label smoothing is performed. Among them, the event intensity level is divided into multiple levels, that is, multiple labels. A degree of abnormality is defined for each label, and the probability prediction is converted into a regression prediction problem.
3. The crisis event prediction method based on the improved neural Hawkes process according to claim 2 is characterized in that: The specific process of step 102 is as follows: The Hawkes process is used to describe the conditional intensity of the event propagation process, and the event m k The conditional intensity function of the i-th cascade sequence at time t for: In the formula, θ represents the parameter set, w h , w Φ and w b They represent the learnable parameters, Represents event m k Features, It is used to measure the impact of the forwarding event that occurred earlier in the sequence on the strength of the condition at the future moment, h(t j ) represents t j The hidden state variable at the moment, g(tt j ) represents the end of time t j The impact of the forwarding behavior on the condition strength at time t, It is used to measure the impact of other forwarding sequences of the same event on this forwarding sequence. The expression of Φ(t) is as follows: Among them, R k Represents event m k Number of sequences sent, t c Represents the cth cascade sequence The last forwarding moment, Indicates the basic strength of the cascade sequence, with event m k The initial characteristics To measure.
4. The crisis event prediction method based on the improved neural Hawkes process according to claim 3 is characterized in that: The specific process of step 103 is as follows: Each event node is defined as the center of the public opinion field, ending at the observation time t o The propagation path of the event is defined as the public opinion field centered on the event node; the conditional intensity functions of all propagation paths of each event are summed to represent the public opinion field energy of the event; the event node m is used as the center of the public opinion field. k The public opinion field formed as the center k Its energy is defined as: Where, t i is the last forwarding time in the i-th cascade sequence.
5. The crisis event prediction method based on the improved neural Hawkes process according to claim 4 is characterized in that: The specific process of step 104 is as follows: (1) Normalize the energy of all public opinion fields: E(P k )←E(P k ) / ∑E(P k ), and then use the graph attention mechanism to update the event node, that is, the representation of the central node of the public opinion field; (2) For friendship relationships between users, we use the graph attention mechanism to aggregate events between nodes; for forwarding relationships between users, we use the graph attention mechanism to aggregate events between nodes; for interest relationships between users and events, we combine the public opinion field energy and the graph attention mechanism to aggregate features between users and events; (3) All the multiple representations of the user are concatenated and then transformed to obtain the final node representation of the user; (4) Repeat steps (1) to (3) twice to obtain the final node embedding of the event node.
6. The crisis event prediction method based on the improved neural Hawkes process according to claim 5, characterized in that: The specific process of step 105 is as follows: In order to predict the level of events in the next week, we first average the embedding vectors of multiple events in each week to obtain a weekly embedding vector of size d for that week, and then concatenate the embedding vectors of the four weeks together to form an input matrix H∈R b×4×d ; Where b represents the batch size fed into the model; A time coding matrix T is introduced to encode the sequence of events. Specifically, the start time of each week is first converted to a Unix timestamp, then normalized, and finally mapped to the same dimension as the text features through a multi-layer perceptron (MLP) to obtain the time coding matrix T. Use H and T to implement the self-attention mechanism in Transformer: Q=W Q (H+f t (T)) K=W K (H+f t (T)) V=W V (H+f t (T)) Among them, W Q 、W K and W V Both represent weight conversion matrices, f t is a parameter matrix used to convert the timestamp matrix to the same size as the embedding matrix; After multi-layer Transformer processing, the event embedding matrix H′ for four weeks is obtained. The last time step, that is, the output of the current week, is used as the representation of the sequence to capture the latest state of the sequence. After obtaining the overall representation, we first use a multi-layer perceptron (MLP) to predict the probability of abnormal events occurring in the next week: in, Represents the predicted probability value of abnormal events occurring in the next week, and then calculates the MSE loss function with the actual probability: Where C1 and Represent the true probability and predicted probability of all samples in the training set respectively; Then, a multi-layer perceptron (MLP) is used to perform category prediction and obtain the prediction results of the event intensity level: in, It is the probability distribution of intensity level of a single sample with a length of 5. The category represented by the position with the largest probability is the predicted intensity category; 0, 1, 2, 3, and 4 represent the five event intensity levels respectively; finally, the intensity is calculated using the cross entropy loss function Among them, C2 and are the true categories and predicted categories of all samples in the training set, respectively.
7. The crisis event prediction method based on the improved neural Hawkes process according to claim 6, characterized in that: The specific process of step 106 is as follows: Add the losses of the two tasks to perform multi-task learning: L=μL1+ρL2 Among them, μ and ρ represent the weights of the anomaly probability prediction loss function and the intensity level loss function, respectively.