A news event prediction method based on recursive bi-hypergraph neural network

Abstract

The application discloses a news event prediction method based on a recursive double hypergraph neural network, and comprises the following steps: extracting news events and constructing a training data set; selecting samples to construct an entity graph and a relation graph; generating an entity hypergraph and a relation hypergraph by using a double hypergraph learning module; performing information aggregation and representation fusion on the entity hypergraph and the relation hypergraph by using a double hypergraph message passing network; combining the fused representation with the initial representation by using a residual gate to generate the input of the next time step; inputting the representation fused at different time steps into a time-dependent encoder based on a self-attention mechanism to obtain a comprehensive representation; predicting all possible future relationships between entities by using a convolution-based knowledge graph model, calculating a loss function to adjust model parameters; and training the model by using all batches of samples, and using the trained knowledge graph model for news event prediction. The method has wide application prospects in the fields of international event prediction, crisis warning and the like.

Description

Technical Field

[0001] This invention belongs to the field of news event prediction technology, specifically relating to a news event prediction method based on a recursive dual hypergraph neural network. Background Technology

[0002] The Global Database of Events, Language, and Tone (GDELT) and the Integrated Crisis Early Warning System (ICEWS) contain a large amount of time-stamped knowledge, i.e., events. Events are represented in the form of a four-tuple (head entity, relation, tail entity, timestamp). Entities include citizens, celebrities, schools, companies, countries, etc., and relations include cooperation, providing aid, refusal, issuing statements, etc. The news event prediction task, based on historical events, maps entities and relations into low-dimensional, dense real-valued vector representations, and then predicts possible future relationships between entity pairs through vector operations. It has broad application prospects in fields such as international event prediction and crisis early warning.

[0003] Early news event prediction methods primarily used shallow encoders such as hyperplane projection and fixed-format encoding to model timestamps, neglecting temporal dependencies. Deep neural network-based methods combine Graph Neural Networks (GNNs) with Recurrent Neural Networks (RNNs) and their variants to model the graph structure information and temporal dependencies in news events, achieving promising results in event prediction tasks. In these methods, the graph is constructed based on quadruples, and GNNs are used to capture pairwise correlations between entities.

[0004] Chinese patent application CN112633483A discloses a method, apparatus, device, and medium for event prediction using a quadruplet gate graph neural network. This method constructs an event graph from multiple initial background events and multiple candidate events, representing the vectors of all events in the event graph as quadruplets. It uses a quadruplet gate graph neural network to perform graph network computation on the event graph and utilizes an attention neural network to compute the event vectors. By scoring the overall vector against each new candidate event vector, the method selects the candidate event corresponding to the candidate event vector with the highest score as the prediction result. However, this method does not consider time dependence.

[0005] Chinese patent application CN115907144A discloses an event prediction method, apparatus, terminal device, and storage medium. This method is based on historical events, selects hyperbolic space as the embedding space to construct an event prediction model, adds the self-attribute information of the person to the static knowledge graph of the person, integrates all relevant information of each person entity, and, based on the generated static information of the person, utilizes the advantages of multi-head self-attention mechanism in processing sequential data to gradually combine the static knowledge graphs of the person at adjacent time points for reasoning, reflecting the interconnectedness between events. However, this method is not applied to the judgment of the person's future activities.

[0006] Recent methods attempt to introduce quadruple-derived structures, such as paths, entity groups, and communities, to capture potential pairwise correlations between entities that are geographically distant or even unreachable. However, these methods fail to capture higher-order correlations between entities—that is, correlations between three, four, or more entities. Furthermore, they also fail to capture higher-order correlations between relationships. Summary of the Invention

[0007] In view of the above, the purpose of this invention is to solve the problem of effectively modeling higher-order correlations between entities and relationships in news event prediction methods. It constructs entity graphs and relationship graphs based on events occurring at each time step, generates entity hypergraphs and relationship hypergraphs using a dual hypergraph learning module, and introduces a dual hypergraph message passing network for representation learning. Based on a recursive dual hypergraph neural network, the representations are combined and further a comprehensive representation is generated. This method can model higher-order correlations between entities and relationships, and predict all possible future relationships between entities. The method of this invention has broad application prospects in fields such as international event prediction and crisis early warning.

[0008] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0009] This invention provides a news event prediction method based on a recurrent dual hypergraph neural network, comprising the following steps:

[0010] Step (1): Preprocess the news event data, extract events, and construct a training dataset;

[0011] Step (2): Construct an entity graph and a relationship graph based on the events that occur at each time step;

[0012] Step (3): In the dual hypergraph learning module, the entity hypergraph mapper and the relation hypergraph mapper are used to capture the higher-order correlations between entities in the entity graph and between relations in the relation graph, respectively, to generate the entity hypergraph and the relation hypergraph.

[0013] Step (4): In the dual hypergraph message passing network, information aggregation on the entity hypergraph and relation hypergraph is performed by using the entity hypergraph neural network and the relation hypergraph neural network respectively. The entity-to-relation fusion machine and the relation-to-entity fusion machine are used to fuse the entity hypergraph and relation hypergraph to obtain the fused entity and relation representations.

[0014] Step (5): Use residual gates to combine the output fused entity and relation representations with the initial entity and relation representations to generate the input for the next time step;

[0015] Step (6) inputs the entity and relation representations fused at different time steps into a time-dependent encoder based on a self-attention mechanism to obtain a comprehensive entity and relation representation;

[0016] Step (7): Based on the comprehensive entity and relation representations, predict all possible future relations between entities using a convolution-based knowledge graph model, and calculate the loss function to adjust the model parameters;

[0017] Step (8): Train the knowledge graph model based on the training dataset, and use the trained knowledge graph model to predict the input samples.

[0018] Further, step (1) includes: extracting events from the news event data, with each event represented as a quadruple, namely, head entity, relation, tail entity, and timestamp; dividing the processed news event data using a sliding time window of size T to obtain a training dataset, wherein each sample contains events occurring at T time steps related to the entities in the news event data; dividing the training dataset into N batches and using them for training in batches, and randomly initializing the learnable entity and relation representations e. i and r i , where i is the sequence number of the i-th entity or relation representation in each batch.

[0019] Further, step (2) includes: constructing an entity graph G based on the events occurring at each time step. e =(V e E e ), where V e For a set of nodes, representing an entity, E e The edge set represents the relationships between entities; the relationships on the entity graph are constructed by treating them as nodes to build the relationship graph G. r =(V r E r ), where V r For a set of nodes, representing a relation, E r For edge sets, based on entity graph G e The relationship is defined by whether entities are shared between them.

[0020] Further, step (3) includes: capturing higher-order correlations between entities using the entity hypergraph mapper in the dual hypergraph learning module to generate an entity hypergraph G. eh =(V e E eh I eh ), where V e E is the set of nodes in the entity hypergraph. eh Let I be the set of entity hyperedges in the entity hypergraph. eh This is an indicator matrix for the relationships between nodes and hyperedges in the entity hypergraph;

[0021] The relation hypergraph mapper in the dual hypergraph learning module is used to capture higher-order correlations between relations, generating a relation hypergraph G. rh =(V r E rh I rh ), where V r Let E be the set of nodes in the relational hypergraph. rh Let I be the set of relation hyperedges in a relation hypergraph. rh It is an indicator matrix for the connections between nodes and hyperedges in the relational hypergraph.

[0022] Preferably, the indicator matrix I in the learning entity hypergraph represents the connections between nodes and hyperedges. eh And the indicator matrix I of the relationships between nodes and hyperedges in the relation hypergraph. rh At this time, a low-rank factorization strategy is introduced to reduce model parameters, and the calculation formula is as follows:

[0023] I eh =E e W eh

[0024] I rh =E r W rh

[0025] Among them, W eh and W rh For learnable parameters, and These are the representation matrices for entities and relations, respectively. Let N be the set of all real numbers. e and N r These are the entity hypergraphs G and G. eh The number of entities and relationships in the hypergraph G rh The number of relations in the equation, where d is the dimension representing the relation.

[0026] Introducing a sparse thresholding strategy makes the indicator matrix I eh and I rhTo reduce the impact of noise, the sparsity is reduced by using the sparsemax function to adaptively retain values ​​above a threshold and truncate other values ​​to 0.

[0027] Furthermore, step (4) includes the following steps:

[0028] 4a) Information aggregation of entities or relations to hyperedges: The representation of an entity or relation hyperedge is obtained by aggregating the entity or relation representations belonging to the corresponding hyperedge. The calculation formula is as follows:

[0029]

[0030]

[0031] in, For entity hyperedge set E eh The representation of the j-th entity superedge in the middle, To make the i-th entity e i Assigned to the j-th entity superedge The probability, For relational hyperedge set E rh The representation of the j-th relation superedge in the equation. To make the i-th relation r i Assigned to the j-th relation superedge The probability, |V e |For entity set V e Number of entities in the middle, |V r |For the set of relations V r Number of relations in China, e i and r i The representation of the i-th entity and relation is initialized randomly;

[0032] 4b) Information aggregation from hyperedges to entities or relations: The hyperedge information corresponding to entities or relations is aggregated. An aggregation operation based on a self-attention mechanism is used to distinguish the influence of different hyperedges. The calculation formula is as follows:

[0033]

[0034]

[0035] in, Let i be the i-th entity in the aggregation. Let Φ be the i-th relation representation of the aggregation. e and Φ r The aggregation function is implemented by a multilayer perceptron, |E eh |For entity hypergraph G eh Medium Entity Hyperedgeset E eh The number of entity hyperedges, |E rh|For the relational hypergraph G rh China's relationship super-edge set E rh The number of relational superedges, Let the attention weights be the super-edge from the i-th entity to the j-th entity. Let the attention weights be the superedges from the i-th relation to the j-th relation. and The calculation formula is as follows:

[0036]

[0037]

[0038]

[0039]

[0040] Among them, W1, W2, W3, and W4 are learnable parameters. The activation function is implemented using LeakyReLU, and || represents the concatenation operation. Let be the similarity metric coefficient between the hyperedge from the i-th entity and the j-th entity. Let N be the similarity metric coefficient of the hyperedge from the i-th relation to the j-th relation. eh and N rh These are the entity hypergraphs G and G. eh and relational hypergraph G rh The number of entity hyperedges and relation hyperedges in the middle;

[0041] 4c) After obtaining the aggregated information, the entity and relation representations are updated, and the calculation formula is as follows:

[0042]

[0043]

[0044] in, This represents the updated i-th entity. Let ξ be the updated representation of the i-th relation. e and ξ r The transformation function is implemented by a multilayer perceptron;

[0045] 4d) First, perform the fusion of relation-to-entity representations, then perform the fusion of entity-to-relation representations. The formula for calculating the fused entity or relation representation is as follows:

[0046]

[0047]

[0048] Among them, e′i Let r′ be the representation of the i-th fused entity. i Let Ψ be the relation representation of the i-th fusion. e and Ψ r The fusion function is implemented using a multilayer perceptron. Let be the entity-relationship indicator matrix, where |ε| is the number of entities in the entity set ε of all event quadruples. The set of relations in all event quadruples Number of relationships For the set of all real numbers, the entity-relationship indicator matrix encodes the relationships between entities and relations on an entity graph, and is defined as follows: if entity e i With relation r i Connect, then M i，j =1, otherwise 0, M i，j This refers to the value in the i-th row and j-th column of the entity-relationship indicator matrix; M aggregates the representations of relationship updates connected to the corresponding entities. i，j e′ j The aggregated representation of entities connected to the correspondence is used, where i∈{1,...,N}. e}, j∈{1, ..., N r}

[0049] Further, step (5) includes: the residual gate determines the proportion of residual information to be retained, which combines the fused entity and relation representations output at time step t with the initial representation to generate the input of the model at time step t+1; for time steps t = 1, 2, ..., T-1, steps (3) to (5) are performed iteratively.

[0050] Further, step (6) includes: introducing a time-dependent encoder based on a self-attention mechanism, using timestamps through position embedding, and generating a timestamp-based position representation using a sine function; using a position-enhanced self-attention mechanism to capture the time dependencies between the fused representations, and outputting a comprehensive entity representation. The relationship between and synthesis

[0051] Further, step (7) includes: taking the comprehensive entity and relation representations as input, using the convolution-based knowledge graph model ConV-TransE to predict all possible relations between entities at the T-th time step, and using the cross-entropy function to calculate the prediction loss for a single sample. Predicted loss based on all samples in a batch Adjust the learnable parameters throughout the model.

[0052] Further, step (8) includes: repeating steps (2) to (7) until all batches of the training dataset have participated in model training and the specified number of iterations has been reached; inputting the processed news event data samples into the trained knowledge graph model for prediction, obtaining the probability of all possible future relationships between given entity pairs, and realizing news event prediction.

[0053] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0054] (1) The method of the present invention introduces a dual hypergraph learning module, which discovers the higher-order correlations between entities and relationships in a parameterized manner.

[0055] (2) The method of the present invention introduces a dual hypergraph message passing network to perform information aggregation and representation fusion on entity hypergraph and relation hypergraph, and can utilize the high-order correlation between entities and relations for representation learning.

[0056] (3) By using residual gates and time-dependent encoders based on self-attention mechanism to recursively fuse the representations of the previous time step and the initial time step and finally generate a comprehensive representation, the prediction accuracy is improved. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is an overall flowchart of a news event prediction method based on a recursive dual hypergraph neural network provided in an embodiment of the present invention;

[0059] Figure 2 This is an overall framework diagram of a news event prediction method based on a recursive dual hypergraph neural network provided in an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0061] The inventive concept of this invention is as follows: Addressing the problem that existing news event prediction methods cannot capture higher-order correlations between entities and relationships, this invention provides a news event prediction method based on a recursive dual hypergraph neural network. It utilizes a dual hypergraph learning module to generate entity hypergraphs and relation hypergraphs, introduces a dual hypergraph message passing network for representation learning, and combines these representations based on the recursive dual hypergraph neural network to further generate a comprehensive representation. This method can model higher-order correlations between entities and relationships, and predict all possible future relationships between entities.

[0062] Figure 1 This is an overall flowchart of a news event prediction method based on a recursive dual hypergraph neural network provided in an embodiment of the present invention. Figure 2 This is an overall framework diagram of a news event prediction method based on a recursive dual hypergraph neural network provided in an embodiment of the present invention. For example... Figure 1 and Figure 2 As shown in the figure, the embodiment provides a news event prediction method based on a recurrent dual hypergraph neural network, including the following steps:

[0063] Step (1) Preprocess the news event data, extract the events, and construct the training dataset.

[0064] 1a) Preprocess the news event data to extract events. The news event data contains a large amount of time-stamped knowledge, which is then extracted. Events are represented by quadruples (e... s ,r,e o The form of τ is used, where e s r is the head entity, r is the relation, and e is the head entity. o Let be the tail entity, τ be the timestamp, and ε be the entity set, mainly including entities such as citizens, celebrities, schools, companies, and countries, and e s e o ∈ε; This is a set of relationships, primarily including those involving cooperation, provision of assistance, refusal, and issuing statements. It is a set of timestamps, and

[0065] The timestamps of the four-tuples represent the timestamps of the events. The four-tuples are arranged chronologically, and then the processed data is divided using a sliding time window of size T to obtain the training dataset. Let T be the set of events occurring at time step t, where each sample contains the set of events occurring at T time steps for the relevant entities in the news data.

[0066] 1b) The training dataset is divided into batches according to a fixed batch size, with a total number of batches of N.

[0067] 1c) Randomly initialize learnable entity and relation representations e i and r i , where i is the sequence number of the i-th entity or relation representation in each batch.

[0068] 1d) Select a batch of training samples with index k sequentially from the training dataset, where k∈{1,...,N}. Repeat steps (2) to (7) for each training sample in the batch.

[0069] Step (2): Construct an entity graph and a relationship graph based on the events that occur at each time step.

[0070] 2a) Construct an entity graph G based on the events that occur at each time step in the historical window [1:T-1]. e =(V e E e ), where V e For a set of nodes, representing an entity, E e An edge set represents the relationships between entities.

[0071] Entity graph G e The nodes are entities, and the edge construction rules are as follows:

[0072]

[0073] in, For the entity graph and the i-th entity e i and the j-th entity e j Related edges.

[0074] 2b) Construct a relationship graph G based on the events that occur at each time step in the historical window [1:T-1]. r =(V r E r ), where V r For a set of nodes, representing a relation, E r For edge sets, based on entity graph G e The relationship is defined by whether entities are shared between them.

[0075] Relationship diagram G r The nodes are relationships on the entity graph, and the edge construction rules are as follows:

[0076]

[0077] in, For the relationship graph with the i-th relation r i and the j-th relation r j Related edges.

[0078] Step (3): In the dual hypergraph learning module, the entity hypergraph mapper and the relation hypergraph mapper are used to capture the high-order correlations between entities in the entity graph and between relations in the relation graph, respectively, to generate the entity hypergraph and the relation hypergraph.

[0079] 3a) The entity hypergraph mapper in the dual hypergraph learning module is used to capture higher-order correlations between entities and generate an entity hypergraph G. eh =(V e E eh I eh ), where V e E is the set of nodes in the entity hypergraph. eh Let I be the set of entity hyperedges in the entity hypergraph. eh It is an indicator matrix for the relationships between nodes and hyperedges in the entity hypergraph.

[0080] 3b) Using the relational hypergraph mapper in the dual hypergraph learning module to capture higher-order correlations between relations, a relational hypergraph G is generated. rh =(V r E rh I rh ), where V r Let E be the set of nodes in the relational hypergraph. rh Let I be the set of relation hyperedges in a relation hypergraph. rh It is an indicator matrix for the connections between nodes and hyperedges in the relational hypergraph.

[0081] In steps 3a) and 3b), the learnable dependencies between entities and entity hyperedges are captured by the entity hypergraph mapper, and formally defined as follows: Where N e and N eh These are the entity hypergraphs G and G. eh The number of entities and entity hyperedges; learnable dependencies between relations and relation hyperedges are captured by a relation hypergraph mapper, formally defined as... Where N r and N rh These are relational hypergraphs G rh The number of relations and relation hyperedges. The entity and relation hypergraph mapper assigns each entity or relation to multiple hyperedges with varying probabilities, with a total probability equal to 1.

[0082] Learning dense indicator matrices will significantly increase the computational cost of the model. To reduce the number of model parameters, we learn the indicator matrices I representing the connections between nodes and hyperedges in the entity hypergraph. eh And the indicator matrix I of the connections between nodes and hyperedges in the relation hypergraph. rh When a low-rank factorization strategy is introduced, the calculation formula is as follows:

[0083] I eh =Ee W eh (3)

[0084] I rh =E r W rh (4)

[0085] Among them, W eh and W rh For learnable parameters, and These are the representation matrices for entities and relations, respectively. Let N be the set of all real numbers. e and N r These are the entity hypergraphs G and G. eh The number of entities and relationships in the hypergraph G rh The number of relations in the equation, where d is the dimension representing the relation.

[0086] To reduce the impact of noise and make the model more robust, a sparse thresholding strategy is introduced to make the indicator matrix I... eh and I rh It becomes sparse, adaptively retaining values ​​above a threshold using the sparsemax function and truncating other values ​​to 0. The threshold is determined by the indicator matrix I. eh and I rh Calculated.

[0087] Step (4): In the dual hypergraph message passing network, information aggregation on the entity hypergraph and relation hypergraph is performed using the entity hypergraph neural network and the relation hypergraph neural network, respectively. The entity-to-relation fusion machine and the relation-to-entity fusion machine are used to fuse the representations of the entity hypergraph and the relation hypergraph to obtain the fused entity and relation representation e′. i and r′ i .

[0088] To utilize the higher-order correlations between entities and between relations, respectively, an entity hypergraph neural network and a relation hypergraph neural network were designed. By aggregating information from entities or relations to hyperedges in step 4a) and from hyperedges to entities or relations in step 4b), representations of entities and relations are obtained.

[0089] 4a) Information aggregation of entities or relations to hyperedges. In this stage, the representation of an entity or relation hyperedge is obtained by aggregating the entity or relation representations belonging to the corresponding hyperedge. The calculation formula is as follows:

[0090]

[0091]

[0092] in, For entity hyperedge set Eeh The representation of the j-th entity superedge in the middle, To make the i-th entity e i Assigned to the j-th entity superedge The probability, For relational hyperedge set E rh The representation of the j-th relation superedge in the equation. To make the i-th relation r i Assigned to the j-th relation superedge The probability, |V e |For entity set V e Number of entities in the middle, |V r |For the set of relations V r Number of relations in China, e i and r i The i-th entity and relation representation is randomly initialized and obtained from step 1c).

[0093] 4b) Information aggregation from hyperedges to entities or relations. In this stage, the hyperedge information corresponding to entities or relations is aggregated. An aggregation operation based on a self-attention mechanism is used to distinguish the influence of different hyperedges. The calculation formula is as follows:

[0094]

[0095]

[0096] in, Let i be the i-th entity in the aggregation. Let Φ be the i-th relation representation of the aggregation. e and Φ r The aggregation function is implemented by a multilayer perceptron, |E eh |For entity hypergraph G eh Medium Entity Hyperedgeset E eh The number of entity hyperedges, |E rh |For the relational hypergraph G rh China's relationship super-edge set E rh The number of relational superedges, Let the attention weights be the super-edge from the i-th entity to the j-th entity. Let the attention weights be the superedges from the i-th relation to the j-th relation. and The calculation formula is as follows:

[0097]

[0098]

[0099]

[0100]

[0101] Among them, W1, W2, W3, and W4 are learnable parameters. The activation function is implemented using LeakyReLU, and || represents the concatenation operation. Let be the similarity metric coefficient between the hyperedge from the i-th entity and the j-th entity. Let N be the similarity metric coefficient of the hyperedge from the i-th relation to the j-th relation. eh and N rh These are the entity hypergraphs G and G. eh and relational hypergraph G rh The number of entity hyperedges and relation hyperedges in the middle.

[0102] 4c) After obtaining the aggregated information, the entity and relation representations are updated, and the calculation formula is as follows:

[0103]

[0104]

[0105] in, This represents the updated i-th entity. Let ξ be the updated representation of the i-th relation. e and ξ r The transformation function is implemented by a multilayer perceptron.

[0106] 4d) To jointly utilize the information from the entity and relation hypergraph, an entity-to-relation fusion machine and a relation-to-entity fusion machine were designed, performing representation fusion under the guidance of the entity-relation indicator matrix. Relation-to-entity representation fusion was performed first, followed by entity-to-relation representation fusion. The formula for calculating the fused entity or relation representation is as follows:

[0107]

[0108]

[0109] Among them, e′ i Let r′ be the representation of the i-th fused entity. i Let Ψ be the relation representation of the i-th fusion. e and Ψ r The fusion function is implemented using a multilayer perceptron. Let be the entity-relationship indicator matrix, where |ε| is the number of entities in the entity set ε of all event quadruples. The set of relations in all event quadruples Number of relationships For the set of all real numbers, the entity-relationship indicator matrix encodes the relationships between entities and relations on an entity graph, and is defined as follows: if entity e i With relation r i Connect, then M i，j =1, otherwise 0, M i，j This refers to the value in the i-th row and j-th column of the entity-relationship indicator matrix; M aggregates the representations of relationship updates connected to the corresponding entities. i，j e′ j The aggregated representation of entities connected to the correspondence is used, where i∈{1,...,N}. e}, j∈{1, ..., N r}

[0110] Step (5) uses residual gates to combine the output fused entity and relation representations with the initial entity and relation representations to generate the input for the next time step.

[0111] In this step, residual gates are used to generate the input X of the model at time step t+1. t+1 It combines the fused entity and relation representations output at time step t with the initial representation, and the calculation formula is as follows:

[0112]

[0113] in, For dot product operation, X t X is the fused entity or relation representation output at time step t. (0) For the initial representation of an entity or relation; S t+1 As a residual gate, it determines the proportion of residual information to be retained, and the calculation formula is as follows:

[0114] S t+1 =σ(W5X t +b) (18)

[0115] Where σ is the sigmoid function, and W5 and b are learnable parameters.

[0116] For time steps t = 1, 2, ..., T-1, steps (3) to (5) are executed iteratively.

[0117] Step (6) involves inputting the entity and relation representations fused from different time steps into a time-dependent encoder based on a self-attention mechanism to obtain a comprehensive entity and relation representation. and

[0118] Compared to RNNs and Long Short-Term Memory (LSTM) networks, position-enhanced self-attention mechanisms exhibit superior performance in sequence modeling. A time-dependent encoder based on a self-attention mechanism is introduced to model the temporal dependencies between representations, utilizing timestamps through position embedding. Timestamp-based positional representations are generated using a sine function, calculated as follows:

[0119]

[0120] Where, {ω i} i＝1，2，...，d Here, τ represents the learnable parameters, d represents the dimension of the entity and relation representations, and τ represents the dimensionality. t For the timestamp corresponding to time step t, use z t =μ t ||X t To represent entities or relations based on location-enhanced representations, given a representation sequence z 1：T-1 ={z1, z2, ..., z t , ..., z T-1}, utilizing a position-enhanced self-attention mechanism to capture the temporal dependencies between fused representations, outputting a comprehensive entity representation (formally defined as} ), and the comprehensive relational representation (formally defined as It integrates time information and graph structure information.

[0121] Step (7): Based on the comprehensive entity and relation representations, predict all possible future relationships between entities using a convolution-based knowledge graph model, and calculate the loss function to adjust the model parameters.

[0122] 7a), given the head entity e s Combined tail entity e o The event prediction task aims to predict relations at time step T based on events in a historical window [1:T-1]. In this step, a convolutional knowledge graph model, Conv-TransE, is used to predict the probability of each relation between entity pairs. The calculation formula is as follows:

[0123]

[0124] in, For the set of events in the history window [1:T-1], e s and e o It consists of a head entity and a tail entity. Let be the probability vector of the relationship. Let σ be the number of relations in the event quadruple, and σ be the sigmoid function. and For a comprehensive representation of the head and tail entities, L r This is a comprehensive relational representation matrix, where each row corresponds to a comprehensive relational representation. ConvFC(·) is a one-dimensional convolutional layer and a fully connected layer.

[0125] 7b) Calculate the cross-entropy loss for a single sample based on the prediction results of all relationships. The calculation formula is as follows:

[0126]

[0127] Where, N r For relational hypergraph G rh The number of relations in y i，j Let y be the label for the relationship between sample i and j. i，j ∈{0,1},P i，j The probability that sample i will have relation j is calculated by formula (20).

[0128] 7c), based on the prediction loss of all samples in the batch Adjust the learnable parameters throughout the model.

[0129] Calculate the prediction loss for all samples in the batch. The specific calculation method is as follows:

[0130]

[0131] in, Let F be the prediction loss for the v-th sample in the batch, and F be the number of samples in each batch.

[0132] Step (8): Train the knowledge graph model based on all samples in all batches, and use the trained knowledge graph model to predict the input samples.

[0133] In this step, repeat steps (2) to (7) until all batches of the training dataset have participated in model training and the specified number of iterations has been reached; input the processed news event data samples into the trained knowledge graph model for prediction, obtain all possible future relationships between given entity pairs, calculate the probability of all relationships between given entity pairs using formula (20), and realize news event prediction.

[0134] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A news event prediction method based on a recurrent dual-hypergraph neural network, characterized in that, Includes the following steps: Step (1): Preprocess the news event data, extract events, and construct a training dataset; Step (2): Construct an entity graph and a relationship graph based on the events that occur at each time step; Step (3): In the dual hypergraph learning module, the entity hypergraph mapper and the relation hypergraph mapper are used to capture the higher-order correlations between entities in the entity graph and between relations in the relation graph, respectively, to generate the entity hypergraph and the relation hypergraph. Step (4): In the dual hypergraph message passing network, information aggregation on the entity hypergraph and relation hypergraph is performed by using the entity hypergraph neural network and the relation hypergraph neural network respectively. The entity-to-relation fusion machine and the relation-to-entity fusion machine are used to fuse the entity hypergraph and relation hypergraph to obtain the fused entity and relation representations. Step (5): Use residual gates to combine the output fused entity and relation representations with the initial entity and relation representations to generate the input for the next time step; Step (6) inputs the entity and relation representations fused at different time steps into a time-dependent encoder based on a self-attention mechanism to obtain a comprehensive entity and relation representation; Step (7): Based on the comprehensive entity and relation representations, predict all possible future relations between entities using a convolution-based knowledge graph model, and calculate the loss function to adjust the model parameters; Step (8): Train the knowledge graph model based on the training dataset, and use the trained knowledge graph model to predict the input samples.

2. The news event prediction method based on a recurrent dual hypergraph neural network according to claim 1, characterized in that, Step (1) includes: Events are extracted from news event data, and each event is represented as a four-tuple: head entity, relation, tail entity, and timestamp. The processed news event data is divided into training datasets using a sliding time window of size T, where each sample contains events occurring at T time steps related to the news event data. The training dataset is then divided into N batches and used for training in batches, with learnable entity and relation representations e randomly initialized. i and r i , where i is the sequence number of the i-th entity or relation representation in each batch.

3. The news event prediction method based on a recurrent dual hypergraph neural network according to claim 1, characterized in that, Step (2) includes: Construct an entity graph G based on the events that occur at each time step. e =(V e E e ), where V e E is a set of nodes, representing an entity. e The edge set represents the relationships between entities; the relationships on the entity graph are constructed by treating them as nodes to build the relationship graph G. r =(V r E r ), where V r For a set of nodes, representing a relation, E r For edge sets, based on entity graph G e The relationship is defined by whether entities are shared between the parties.

4. The news event prediction method based on a recurrent dual hypergraph neural network according to claim 1, characterized in that, Step (3) includes: The entity hypergraph mapper in the dual hypergraph learning module is used to capture higher-order correlations between entities, generating an entity hypergraph G. eh =(V e E eh I eh ), where V e E is the set of nodes in the entity hypergraph. eh Let I be the set of entity hyperedges in the entity hypergraph. eh This is an indicator matrix for the relationships between nodes and hyperedges in the entity hypergraph; The relation hypergraph mapper in the dual hypergraph learning module is used to capture higher-order correlations between relations, generating a relation hypergraph G. rh =(V r E rh I rh ), where V r Let E be the set of nodes in the relational hypergraph. rh Let I be the set of relation hyperedges in a relation hypergraph. rh It is an indicator matrix for the connections between nodes and hyperedges in the relational hypergraph.

5. The news event prediction method based on a recurrent dual hypergraph neural network according to claim 4, characterized in that, In learning the indicator matrix I of the relationships between nodes and hyperedges in a hypergraph. eh And the indicator matrix I of the relationships between nodes and hyperedges in the relation hypergraph. rh At this time, a low-rank factorization strategy is introduced to reduce model parameters, and the calculation formula is as follows: I eh =E e W eh I rh =E r W rh Among them, W eh and W rh For learnable parameters, and These are the representation matrices for entities and relations, respectively. Let N be the set of all real numbers. e and N r These are the entity hypergraphs G and G. eh The number of entities and relationships in the hypergraph G rh The number of relations in the equation, where d is the dimension representing the relation. Introducing a sparse thresholding strategy makes the indicator matrix I eh and I rh To reduce the impact of noise, the sparsity is reduced by using the sparsemax function to adaptively retain values ​​above a threshold and truncate other values ​​to 0.

6. The news event prediction method based on a recurrent dual hypergraph neural network according to claim 1, characterized in that, Step (4) includes the following steps: 4a) Information aggregation of entities or relations to hyperedges: The representation of an entity or relation hyperedge is obtained by aggregating the entity or relation representations belonging to the corresponding hyperedge. The calculation formula is as follows: in, For entity hyperedge set E eh The representation of the j-th entity superedge in the middle, To make the i-th entity e i Assigned to the j-th entity superedge Γ j eh The probability, For relational hyperedge set E rh The representation of the j-th relation superedge in the equation. To make the i-th relation r i Assigned to the j-th relation superedge Γ j rh The probability, |V e |For entity set V e Number of entities in the middle, |V r |For the set of relations V r Number of relations in China, e i and r i The representation of the i-th entity and relation is initialized randomly; 4b) Information aggregation from hyperedges to entities or relations: The hyperedge information corresponding to entities or relations is aggregated. An aggregation operation based on a self-attention mechanism is used to distinguish the influence of different hyperedges. The calculation formula is as follows: in, Let i be the i-th entity in the aggregation. Let Φ be the i-th relation representation of the aggregation. e and Φ r The aggregation function is implemented by a multilayer perceptron, |E eh |For entity hypergraph G eh Medium Entity Hyperedgeset E eh The number of entity hyperedges, |E rh |For the relational hypergraph G rh China's relationship super-edge set E rh The number of relational superedges, Let the attention weights be the super-edge from the i-th entity to the j-th entity. Let the attention weights be the superedges from the i-th relation to the j-th relation. and The calculation formula is as follows: Among them, W1, W2, W3, and W4 are learnable parameters. The activation function is implemented using LeakyReLU, and || represents the concatenation operation. Let be the similarity metric coefficient between the hyperedge from the i-th entity and the j-th entity. Let N be the similarity metric coefficient of the hyperedge from the i-th relation to the j-th relation. eh and N rh These are the entity hypergraphs G and G. eh and relational hypergraph G rh The number of entity hyperedges and relation hyperedges in the middle; 4c) After obtaining the aggregated information, the entity and relation representations are updated, and the calculation formula is as follows: in, This represents the updated i-th entity. Let ξ be the updated representation of the i-th relation. e and ξ r The transformation function is implemented by a multilayer perceptron; 4d) First, perform the fusion of relation-to-entity representations, then perform the fusion of entity-to-relation representations. The formula for calculating the fused entity or relation representation is as follows: Among them, e′ i Let r′ be the representation of the i-th fused entity. i Let Ψ be the relation representation of the i-th fusion. e and Ψ r The fusion function is implemented using a multilayer perceptron. This is the entity-relationship indicator matrix, where The set of entities in all event quadruples The number of entities, The set of relations in all event quadruples Number of relationships For the set of all real numbers, the entity-relationship indicator matrix encodes the relationships between entities and relations on an entity graph, and is defined as follows: if entity e i With relation r i Connect, then M i,j =1, otherwise 0, M i,j This refers to the value in the i-th row and j-th column of the entity-relationship indicator matrix; M aggregates the representations of relationship updates connected to the corresponding entities. i,j e′ j The aggregated representation of entities connected to the correspondence is used, where i∈{1,…,n} e }, j∈{1,…,N r } 7. The news event prediction method based on a recurrent dual hypergraph neural network according to claim 1, characterized in that, Step (5) includes: The residual gate determines the proportion of residual information to be retained, which combines the fused entity and relation representations output at time step t with the initial representation to generate the input of the model at time step t+1; for time steps t = 1, 2, ..., T-1, steps (3) to (5) are performed iteratively.

8. The news event prediction method based on a recurrent dual hypergraph neural network according to claim 1, characterized in that, Step (6) includes: A temporal dependency encoder based on a self-attention mechanism is introduced. It utilizes timestamps through position embedding and generates timestamp-based positional representations using a sine function. A position-enhanced self-attention mechanism is then employed to capture the temporal dependencies between the fused representations, outputting a comprehensive entity representation. The relationship between and synthesis 9. The news event prediction method based on a recurrent dual hypergraph neural network according to claim 1, characterized in that, Step (7) includes: Using a comprehensive representation of entities and relationships as input, the Conv-TransE convolutional knowledge graph model is used to predict all possible relationships between entities at time step T, and the prediction loss for a single sample is calculated using the cross-entropy function. Predicted loss based on all samples in a batch Adjust the learnable parameters throughout the model.

10. The news event prediction method based on a recurrent dual hypergraph neural network according to claim 1, characterized in that, Step (8) includes: Repeat steps (2) to (7) until all batches of the training dataset have participated in model training and the specified number of iterations has been reached; input the processed news event data samples into the trained knowledge graph model for prediction, obtain the probability of all possible future relationships between a given entity pair, and realize news event prediction.

Images