News event prediction method based on heterogeneous evolutionary event clustering

Abstract

The invention discloses a news event prediction method based on heterogeneous evolutionary event clustering, which comprises the following steps: generating event representations based on preliminarily updated entity representations and relationship representations in a constructed entity graph, and constructing an event graph by taking events as nodes and taking heterogeneous relationships between the events as edges; obtaining event clusters through fuzzy clustering and constructing an event cluster graph; using a self-supervised optimization algorithm to optimize the event cluster representation according to the distance and similarity between the event clusters on the event cluster graph; capturing implicit correlation among the event clusters by using an implicit relation encoder, and sequentially updating representation of the event clusters, representation of events, and representation of entities and relations after sparsification and information aggregation; and predicting through the news event model based on convolution. According to the method, the pairwise correlation, the high-order correlation and the multi-step time sequence evolution of the events are effectively modeled, and the method has important application value in the aspects of international situation analysis, social governance, intelligent decision support and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of news event prediction, and in particular relates to a news event prediction method based on heterogeneous evolution event clustering. Background Art

[0002] News event repositories, such as the Global Database of Events, Language, and Tone (GDELT) and the Integrated Crisis Early Warning System (ICEWS), play a vital role in the modern information society. These massive databases accumulate massive amounts of news event data with precise timestamps, providing a rich source of information for researchers, policymakers, and business analysts. These news events are typically represented as a quadruple (head entity, relationship, tail entity, timestamp). Entities include countries, regions, citizens, companies, and other types of objects, while relationships encompass collaboration, access, statement release, condemnation, and rejection.

[0003] The task of news event prediction is precisely based on this rich historical news event data. Its core goal is to skillfully map entities and relationships into low-dimensional, dense real-valued vectors and use vector calculations to accurately predict possible future relationships between entities. This cutting-edge research direction has demonstrated tremendous application potential and value in a wide range of fields, including international situation analysis, social governance, and intelligent decision support. For example, in international situation analysis, in-depth exploration and prediction of news events between countries can help analysts better grasp the direction of the international situation. In social governance, it can help to promptly identify potential social problems and provide a scientific basis for policy making. In the field of intelligent decision support, news event prediction can provide strong data support for various decisions, improving the scientific nature and accuracy of decision-making.

[0004] Deep learning-based news event prediction methods typically utilize graph neural networks, recurrent neural networks, and their variants to deeply model the graph structure and temporal dependencies of news events, achieving strong performance in news event prediction tasks. Among these methods, graph neural networks can fully leverage the graph structure built on quadruple pairs to capture pairwise correlations between entities and reveal the underlying complex relationships between them. In recent years, with the continuous advancement of research, the latest methods have introduced derived structures such as entity groups, communities, hypergraphs, and evolutionary clusters to further capture higher-order correlations between multiple entities and their temporal evolution, that is, correlations between three or more entities that exist simultaneously. These methods typically rely on decomposing news events into entities and relations for representation learning. While they can reflect the characteristics of events to a certain extent, they cannot fully capture the heterogeneous correlations between events, namely, pairwise correlations, high-order correlations, and their multi-step temporal evolution.

[0005] Therefore, the limitations of existing methods have, to a certain extent, restricted the accuracy and comprehensiveness of news event predictions, and further exploration of more effective news event prediction methods is needed. Summary of the Invention

[0006] In view of the above, the purpose of the present invention is to provide a news event prediction method based on heterogeneous evolutionary event clustering. In order to effectively model the heterogeneous correlations between events, an entity graph is constructed according to the news events occurring at each timestamp, an event graph is constructed using a tensor-based relationship-aware event encoding module, an event cluster graph is constructed using an event-aware multi-step evolutionary clustering module, and an event cluster graph information transmission module is introduced for representation learning to model the pairwise correlations, high-order correlations and multi-step temporal evolution between events, so as to achieve more accurate news event prediction, which has important application value in international situation analysis, social governance and intelligent decision support.

[0007] To achieve the above-mentioned purpose, the present invention provides the following technical solutions:

[0008] An embodiment of the present invention provides a news event prediction method based on heterogeneous evolution event clustering, comprising the following steps:

[0009] Build an entity graph based on news events, and use a relationship-aware graph convolutional neural network to aggregate information on the entity graph to obtain preliminary updated entity representations and relationship representations;

[0010] In the event graph construction module, event representations are generated based on the initially updated entity representations and relation representations through a tensor-based relation-aware event encoding module, and an event graph is constructed based on the event representations by treating events as nodes and heterogeneous relations between events as edges.

[0011] In the event-aware multi-step evolutionary clustering module, a fuzzy clustering algorithm is used to obtain the soft membership matrix of events to event clusters based on the event graph. Event clusters with different historical timestamps are aligned and fused to form an event cluster graph, and a time smoothing loss is constructed based on the distance between event clusters.

[0012] A self-supervised optimization algorithm is used to optimize the event cluster representation based on the distance and similarity between event clusters on the event cluster graph, and a self-supervised smoothing loss is constructed based on the similarity between event clusters before and after optimization.

[0013] In the event cluster graph information transfer module, an implicit relation encoder is used to capture the implicit correlation between event clusters and perform sparsification and information aggregation to update the event cluster representation. The event representation is then updated based on the soft membership matrix of the event to the event cluster, thereby updating the entity and relationship representations.

[0014] The entity representations and relationship representations updated at different timestamps are input into the time-dependent encoder based on the attention mechanism to obtain comprehensive entity representations and relationship representations. All possible future relationships between entities are predicted through the convolution-based news event model. The total prediction loss including prediction loss, temporal smoothing loss and self-supervised smoothing loss is constructed to train the entire network, and the trained news event model is used to predict news events.

[0015] Preferably, generating an event representation based on the preliminary updated entity representation and relationship representation through a tensor-based relationship-aware event encoding module comprises:

[0016] First, the tensor T is used to encode the element-by-element interaction between entity representation and relation representation. The calculation formula is as follows:

[0017]

[0018] Among them, v s,i Represents the interaction v between the head entity s and the relation r s The element v o,i Represents the interaction v between the tail entity o and the relation r o Elements, T ijk Represents the elements of tensor T, subscripts i, j, k represent T ijk The index value in the three directions of the tensor T, h s,j is the jth element of the head entity representation, h o,j is the jth element of the tail entity representation, h r,k is the kth element of the relation;

[0019] Then, we combine the interaction between the head entity and the relationship v s and the interaction between the tail entity and the relationship v o , we get the event representation ev, the calculation formula is as follows:

[0020] ev=W s v s +W o v o

[0021] Among them, W s and W o is a learnable parameter.

[0022] Preferably, the event-based representation constructs an event graph by considering events as nodes and heterogeneous relationships between events as edges, including:

[0023] Based on the event representation ev, the node set of the event graph at timestamp t is represented as Subscript i is the index of the event, using To express events The set of entities contained in in and Represent the head entity and the tail entity respectively;

[0024] Two types of heterogeneous edges, shared entity edges and adjacent edges, are constructed to describe the relationship between events. They are defined as follows:

[0025] Shared Entity Edges If two events with indices i and j share at least one entity, there is a shared entity edge between the two events, represented as:

[0026]

[0027] adjacent edges : If one event is in the k-neighboring set of the other event, then there is a neighboring edge between the two events, which is represented by:

[0028]

[0029] Where kNN(·) represents the k-nearest neighbor algorithm, Indicates equivalence, V indicates or;

[0030] The two heterogeneous edges together constitute the edge set in the event graph, namely Then, an event graph is constructed based on the node set and edge set. And use the relationship-aware graph convolutional neural network to aggregate information on the event graph.

[0031] Preferably, aligning and fusing event clusters with different historical timestamps to construct an event cluster graph includes:

[0032] By aggregating the entities contained in the events in each event cluster, the entity distribution matrix CE of the event cluster at time t is obtained: t , the calculation formula is as follows:

[0033] CE t =(U t ) T M t D -1

[0034]

[0035] Among them, M t represents the event-entity mapping matrix, Indicates M t The i-th element in the event If the entity e is included, its value is 1, otherwise it is 0; D represents a diagonal matrix, each element in D represents the total number of times an entity appears in all events, which is used for normalization; U t represents the soft membership matrix of events to event clusters, Representing an event The set of entities contained in , where the superscript T indicates transposition;

[0036] Subsequently, the event cluster-aware Hungarian matching algorithm is used to ensure smooth alignment of event clusters with different timestamps. It first constructs an affinity matrix a to quantify the similarity between event clusters with different timestamps. For example, if the event clusters with timestamps t-1 and t are aligned, the calculation formula is as follows:

[0037]

[0038] Among them, the element A in the jth row and kth column of A j,k Represents the entity distribution matrix of event cluster j at timestamp t-1 The entity distribution matrix of event cluster k with timestamp t The similarity between them, cos(·) represents the cosine similarity function;

[0039] After the affinity matrix A is constructed, the event cluster-aware Hungarian matching algorithm is used to find the optimal alignment to maximize the sum of similarities between the aligned event clusters. The calculation formula is as follows:

[0040]

[0041] Among them, π represents the permutation function that realizes the smooth alignment of event clusters with different timestamps, π τ (j) represents the index of the event cluster with timestamp τ aligned with the event cluster j with timestamp t, N c represents the total number of event clusters;

[0042] After aligning the event clusters of historical timestamps, the aligned event clusters are average pooled to obtain the fused event cluster representation. The calculation formula is as follows:

[0043]

[0044] Among them, N t Represents the number of timestamps up to and including timestamp t, represents the representation of event cluster j at timestamp t, represents the event cluster of timestamp τ aligned with event cluster j of timestamp t;

[0045] Then, construct the event cluster graph in, The node set representing the timestamp t represents the event cluster after alignment and fusion. The set of edges representing timestamp t represents the implicit correlation between event clusters.

[0046] Preferably, the calculation formula of the time smoothing loss is as follows:

[0047]

[0048] in, represents the temporal smoothing loss.

[0049] Preferably, the optimizing the event cluster representation according to the distance and similarity between event clusters on the event cluster graph using a self-supervised optimization algorithm comprises:

[0050] Based on the Euclidean distance between event clusters, a pairwise distance matrix L is constructed, and a distance weight matrix K is constructed. The calculation formula is as follows:

[0051]

[0052]

[0053] Among them, the element L in the jth row and kth column of L j,k Represents the jth event cluster at timestamp t in the event cluster graph constructed after alignment and fusion and the kth event cluster The Euclidean distance between j,k represents the element in the jth row and kth column of K, and α is a hyperparameter used to control the amplitude of the distance weight adjustment;

[0054] Subsequently, the event cluster similarity matrix F is constructed based on the cosine similarity between event clusters, and the similarity adjustment matrix S is constructed. The calculation formula is as follows:

[0055]

[0056] Among them, F j,k represents the element in row j and column k of F, S j,k represents the element in row j and column k of S, and β is a hyperparameter used to control the magnitude of similarity adjustment;

[0057] The self-supervised optimization of event clusters is achieved by integrating distance weighting and similarity adjustment, and the calculation formula is as follows:

[0058]

[0059] in, represents the event cluster after self-supervised optimization, N c Indicates the total number of event clusters.

[0060] Preferably, the calculation formula of the self-supervised smoothing loss is as follows:

[0061]

[0062] in, represents the self-supervised smoothing loss.

[0063] Preferably, the convolution-based news event model adopts the ConvTransE model.

[0064] Preferably, the calculation formula for the prediction loss is as follows:

[0065]

[0066] in, represents the prediction loss, N r Indicates the number of relationships, y i,j Indicates the label of entity relationship j in sample i, p i,j Represents the predicted probability of entity relationship j in sample i.

[0067] Preferably, the total prediction loss is expressed as:

[0068]

[0069] in, represents the total prediction loss in a training batch, represents the prediction loss of the vth sample in the batch, F represents the number of samples in each batch, represents the time smoothing loss of the v-th sample, represents the self-supervised smoothing loss of the vth sample, and λ and γ represent weights.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] (1) The present invention introduces an event graph construction module, utilizes a tensor-based relation-aware event encoding module to generate event representations, and constructs heterogeneous edges between events to model pairwise correlations between events.

[0072] (2) The present invention introduces an event-aware multi-step evolutionary clustering module, uses a fuzzy clustering algorithm to obtain the soft membership matrix of events to event clusters, and uses an event cluster-aware unsupervised alignment mechanism to align and fuse event clusters with different historical timestamps to construct an event cluster graph, so as to model the high-order correlations between events and their temporal evolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0074] Figure 1 This is an overall flow chart of a news event prediction method based on heterogeneous evolution event clustering provided by an embodiment of the present invention;

[0075] Figure 2 This is an overall framework diagram of the news event prediction method based on heterogeneous evolution event clustering provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0077] The inventive concept of the present invention is: in view of the problem that the existing technology is difficult to effectively model the heterogeneous correlations between events in the news event prediction task, that is, the pairwise correlations, high-order correlations and multi-step temporal evolution between events, the embodiment of the present invention provides a news event prediction method based on heterogeneous evolution event clustering. First, an entity graph is constructed according to the news events occurring at each timestamp, and a relationship-aware graph convolutional neural network is used to perform information aggregation on the entity graph; then, in the event graph construction module, a tensor-based relationship-aware event encoding module is used to generate event representations and construct an event graph; secondly, in the event-aware multi-step evolution clustering module, a fuzzy clustering algorithm is used to obtain the soft membership matrix of events to event clusters, and an event cluster-aware unsupervised alignment mechanism is applied to align and fuse event clusters of different historical timestamps; then, an event cluster graph is constructed based on the aligned and fused event clusters of each timestamp, and a self-supervised optimization algorithm is used to optimize the node representations in the event cluster graph; then, in the event cluster graph information transmission module, an implicit relationship encoding is used to generate event representations. The encoder captures the implicit correlations between event clusters and sparsifies them using a learnable sparse algorithm. A graph neural network is then used to aggregate information on the event cluster graph to update the representation of the event clusters. Event representations are then updated based on the soft membership matrix of the event to the event cluster, further updating the entity and relationship representations. The updated entity and relationship representations are then combined with the input of the previous timestamp via a temporal residual gate as the model input for the next timestamp. The updated entity and relationship representations at different timestamps are then fed into a temporally dependent encoder based on an attention mechanism to obtain a comprehensive entity and relationship representation. Finally, a convolution-based news event model is used to predict all possible future relationships between entities. This approach effectively models pairwise correlations, high-order correlations, and their multi-step temporal evolution between events, helping to further improve the accuracy of news event prediction.

[0078] Figure 1 This is an overall flow chart of a news event prediction method based on heterogeneous evolution event clustering provided by an embodiment of the present invention. Figure 2 This is the overall framework diagram of the news event prediction method based on heterogeneous evolution event clustering provided by the embodiment of the present invention. Figure 1 and Figure 2 As shown, the embodiment provides a news event prediction method based on heterogeneous evolution event clustering, comprising the following steps:

[0079] Step 1: Preprocess the news event data and extract relevant news events. Use a sliding time window of size T to divide the processed data to obtain a training data set, where each sample contains news events that occurred at T timestamps in the news event data.

[0080] News events with timestamps contained in news event data are represented as quadruple (s, r, o, t), where s represents the head entity, r represents the relationship, o represents the tail entity, and t represents the timestamp. s, o∈ε, where ε represents the entity set, mainly including entities such as country, region, citizen, and company. in A collection of representative relationships, mainly including relationships such as cooperation, visits, statements, condemnation and rejection. in Represents a timestamp collection.

[0081] The timestamp of the quadruple is the moment when the news event occurs. The quadruple is arranged in time sequence, and then the processed data is divided using a sliding time window of size T to obtain the training data set. Represents a set of news events that occurred at timestamp t, where each sample contains a set of news events that occurred at T timestamps in the news data

[0082] Step 2: Randomly initialize the learnable entity representation e according to the node’s in-degree i , randomly initialize the learnable relation representation r i , where the subscript i represents the index of the entity representation or relation representation.

[0083] Step 3: Divide the training dataset into batches according to a fixed batch size, with a total number of batches being N.

[0084] Step 4: Sequentially select a batch of training samples with index k from the training dataset, where k∈{1,…,N}. Repeat steps 5-16 for each training sample in the batch.

[0085] Step 5: Build an entity graph based on the news events that occurred at each timestamp in, The set of nodes representing the timestamp t represents the entity; The edge set representing the timestamp t represents the relationship between entities.

[0086] Step 6: for timestamps t=1, 2, ..., T-1, iteratively perform steps 7-13.

[0087] Step 7: Use the relationship-aware graph convolutional neural network to aggregate information on the entity graph.

[0088] In this step, a relationship-aware graph convolutional neural network is used to aggregate information on the entity graph. The calculation formula is as follows:

[0089]

[0090] in, It represents the tail entity representation after information aggregation. Since the tail entity in a news event can also serve as the head entity of another event, it is also regarded as the head entity representation after information aggregation. and They represent the representation of the head entity s, tail entity o and relation r at the lth layer of timestamp t respectively. W1 and W2 are learnable parameters, d o Indicates the in-degree of the tail entity, and RReLU(·) represents the activation function. In the following content, for the sake of brevity, the superscript l of the variable will be omitted without causing ambiguity. The relationship representation of the timestamp It is obtained by the entity representation related to r and the representation of r at the previous timestamp. The calculation formula is as follows:

[0091]

[0092] Among them, [;] represents the concatenation operation, pooling[·] represents the average pooling operation, Represents the entity representation associated with r at timestamp t, where It is understood that i in (i, r, o, t) represents the head entity and i in (s, r, i, t) represents the tail entity.

[0093] Step 8: In the event graph construction module, the tensor-based relation-aware event encoding module is used to generate event representations and construct the event graph. in, The set of nodes representing timestamp t, representing events; The set of edges representing timestamps t represents heterogeneous edges between events.

[0094] In this step, the tensor T is first used to encode the element-by-element interaction between entity representation and relation representation. The calculation formula is as follows:

[0095]

[0096] Among them, v s,i Represents the interaction v between the head entity s and the relation r s The element v o,i Represents the interaction v between the tail entity o and the relation r o Elements, T ijk Represents the elements of tensor T, subscripts i, j, k represent T ijk The index value in the three directions of the tensor T, h s,j is the jth element of the head entity representation, h o,j is the jth element of the tail entity representation, h r,k is the kth element of the relation.

[0097] Then, combining the interaction between the head entity and the relationship and the interaction between the tail entity and the relationship, we get the representation of the event ev, which is calculated as follows:

[0098] ev=W s v s +W o v o (5)

[0099] Among them, W s and W o is a learnable parameter.

[0100] After obtaining the event representation, the node set of the event graph at timestamp t is represented as use To express events The set of entities contained in in and Represent the head entity and the tail entity respectively.

[0101] Since there are no predefined edges between events in news event data, two types of heterogeneous edges are constructed to describe the relationship between events, namely shared entity edges and adjacent edges. The specific definitions are as follows:

[0102] Shared Entity Edges If two events with indices i and j share at least one entity, then there is a shared-entity edge between the two events, which is defined as follows:

[0103]

[0104] adjacent edges If, for two events with indices i and j, one event representation is in the k-neighboring set of the other event representation, then there is a neighboring edge between the two events, which is defined as follows:

[0105]

[0106] Where kNN(·) represents the k-nearest neighbor algorithm, represents equivalence, and ∨ represents or.

[0107] The two heterogeneous edges together constitute the edge set in the event graph, namely Then, construct the event graph And use the relationship-aware graph convolutional neural network to aggregate information on the event graph.

[0108] Step 9: In the event-aware multi-step evolutionary clustering module, the soft membership matrix U of the event to the event cluster and the event cluster of the time stamp t are obtained by the fuzzy clustering algorithm.

[0109] In this step, the soft membership matrix U of the event to event cluster is first obtained by optimizing the following objective function J. The calculation formula is as follows:

[0110]

[0111] Among them, N ev and N c Represent the number of events and event clusters respectively, i represents the index of the event, j represents the index of the event cluster, and the element in the i-th row and j-th column of the soft membership matrix U is Represents an event at timestamp t Event cluster centroid The superscript m represents the fuzzy smoothing hyperparameter, the value of m is set to be greater than 1, <·> represents the dot product operation, and ||·|| represents the modulus of the vector.

[0112] Then we can get the overlapping event clusters of timestamp t, and the calculation formula is as follows:

[0113]

[0114]

[0115] in, represents the representation of event cluster j at timestamp t, which is obtained by weighting the event representation and the event's membership to the event cluster. Represents the cluster centroid The indices other than are the cluster centroids of k.

[0116] Step 10: Apply the event cluster-aware unsupervised alignment mechanism to align and fuse event clusters with different historical timestamps to construct an event cluster graph. in, The set of nodes representing timestamp t represents the event cluster; Represents the edge set of t timestamps, represents the implicit correlation between event clusters, and constructs a time smoothing loss based on the cosine distance between event clusters with different historical timestamps

[0117] In this step, the entity distribution matrix CE of the event cluster is obtained by aggregating the entities contained in the events in each event cluster. t , the calculation formula is as follows:

[0118] CE t =(U t ) T M t D -1 (11)

[0119]

[0120] Among them, M t represents the event-entity mapping matrix, Indicates M t The i-th element in the event If the entity e is included, its value is 1, otherwise it is 0; D represents a diagonal matrix, each element in D represents the total number of times an entity appears in all events, which is used for normalization; U t represents the soft membership matrix of events to event clusters, Representing an event The set of entities contained in , where the superscript T indicates transposition.

[0121] Subsequently, the event cluster-aware Hungarian matching algorithm is used to ensure smooth alignment of event clusters with different timestamps. It first constructs an affinity matrix A to quantify the similarity between event clusters with different timestamps. For example, if the event clusters with timestamps t-1 and t are aligned, the calculation formula is as follows:

[0122]

[0123] Among them, the element A in the jth row and kth column of A j,k Represents the entity distribution matrix of event cluster j at timestamp t-1 The entity distribution matrix of event cluster k with timestamp t The similarity between them, cos(·) represents the cosine similarity function.

[0124] After the affinity matrix A is constructed, the event cluster-aware Hungarian matching algorithm is used to find the optimal alignment to maximize the sum of similarities between the aligned event clusters. The process of finding the optimal alignment can be regarded as an optimization problem, and the calculation formula is as follows:

[0125]

[0126] Among them, π represents the permutation function that realizes the smooth alignment of event clusters with different timestamps, π τ (j) represents the index of the event cluster with timestamp τ aligned with the event cluster j with timestamp t, N c Indicates the total number of event clusters.

[0127] After aligning the event clusters of historical timestamps, the aligned event clusters are average pooled to obtain the fused event cluster representation. The calculation formula is as follows:

[0128]

[0129] Among them, N t Represents the number of timestamps up to and including timestamp t, represents the representation of event cluster j at timestamp t, Denotes the representation of the event cluster at timestamp τ aligned with the event cluster j at timestamp t.

[0130] Then, construct the event cluster graph in, The set of nodes representing timestamp t represents the event cluster; The set of edges representing timestamp t represents the implicit correlation between event clusters.

[0131] In addition, the time smoothing loss is obtained based on the cosine distance between event clusters of historical timestamps, and the calculation formula is as follows:

[0132]

[0133] in, represents the temporal smoothing loss.

[0134] Step 11: Use the self-supervised optimization algorithm to optimize the node representation based on the Euclidean distance and cosine similarity between nodes on the event cluster graph, and construct a self-supervised smoothing constraint based on the node similarity before and after optimization.

[0135] In this step, a pairwise distance matrix L is constructed based on the Euclidean distance between event clusters, and a distance weight matrix K is constructed. The calculation formula is as follows:

[0136]

[0137]

[0138] Among them, the element L in the jth row and kth column of L j,k Represents the jth event cluster at timestamp t in the event cluster graph constructed after alignment and fusion and the kth event cluster The Euclidean distance between j,k represents the element in the jth row and kth column of K, and α is a hyperparameter with a value between 0 and 1, which is used to control the adjustment amplitude of the distance weight.

[0139] Subsequently, the event cluster similarity matrix F is constructed based on the cosine similarity between event clusters, and the similarity adjustment matrix S is constructed. The calculation formula is as follows:

[0140]

[0141]

[0142] Among them, F j,k represents the element in row j and column k of F, S j,k represents the element in the j-th row and k-th column of S, and β is a hyperparameter with a value between 0 and 1, which is used to control the amplitude of the similarity adjustment.

[0143] The self-supervised optimization of event clusters is achieved by integrating distance weighting and similarity adjustment, and the calculation formula is as follows:

[0144]

[0145] in, represents the event cluster after self-supervised optimization, N c Indicates the total number of event clusters.

[0146] In addition, the self-supervised smoothing loss is obtained according to the cosine distance between event clusters before and after self-supervised optimization, and the calculation formula is as follows:

[0147]

[0148] in, represents the self-supervised smoothing loss.

[0149] Step 12: In the event cluster graph information transfer module, the implicit relation encoder is used to capture the implicit correlation between event clusters, and the implicit correlation is sparsely distributed using a learnable sparse algorithm. Then, information aggregation is performed on the event cluster graph through a graph neural network to update the representation of the event cluster, and the event representation is updated based on the soft membership matrix U of the event to the event cluster. Then update the entity representation and relational representation

[0150] In this step, the implicit correlation between event clusters is captured by the implicit relation encoder in the event cluster graph information transfer module. The implicit correlation between the event cluster pair consisting of the i-th event cluster and the j-th event cluster at timestamp t is The calculation formula is as follows:

[0151]

[0152] in, represents the transformation function, which is implemented by the multi-layer perceptron, and ReLU(·) represents the activation function.

[0153] In addition, the strength of the implicit correlation is quantified to obtain The calculation formula is as follows:

[0154]

[0155] Here, σ represents the sigmoid function, which ensures that the resulting implicit correlation strength value is between 0 and 1, and Conv(·) represents the convolution operation.

[0156] Then, the implicit correlation is sparsified by a learnable sparsification algorithm to obtain The calculation formula is as follows:

[0157]

[0158] Here, κ is a learnable parameter.

[0159] Then, information aggregation is performed on the event cluster graph through the graph neural network. The calculation formula is as follows:

[0160]

[0161] in, It is the representation of implicit correlation after intensity quantization.

[0162] Then update the event cluster representation and event representation, and the calculation formula is as follows:

[0163]

[0164] in, represents the updated representation of the i-th event cluster at timestamp t, represents the element in row i and column j of the soft membership matrix, Indicates that the timestamp is N c The updated representation of the jth event cluster among the event clusters, Represents the updated representation of the i-th event at timestamp t.

[0165] After updating the event representation, update the entity representation and relational representation Use E ′ and R ′ Respectively and In the following, for the sake of brevity, the variable subscripts i and j will be omitted.

[0166] In step 13, the updated entity and relation representations output at timestamp t are combined with the entity and relation representations of the input model at timestamp t through a temporal residual gate to generate the input of the model at timestamp t+1.

[0167] In this step, the updated entity and relation representations output at timestamp t are combined with the representations of the input model at timestamp t through a temporal residual gate, which is calculated as follows:

[0168]

[0169] in, represents the dot product operation, is the updated entity or relation representation output at timestamp t, H t-1is the updated entity or relation representation output at timestamp t-1, i.e., the representation of the input model at timestamp t, X t It is the time residual gate, which determines the proportion of information to be retained. The calculation formula is as follows:

[0170] X t =σ(W3H t-1 +b) (30)

[0171] Among them, σ is the sigmoid function, W3 and b are learnable parameters.

[0172] Step 14: Input the updated entity and relationship representations at different timestamps into the time-dependent encoder based on the attention mechanism to obtain the comprehensive entity and relationship representations. and

[0173] In this step, a time-dependent encoder based on the attention mechanism is introduced to model the temporal dependency between representations. The calculation formula of the representation enhanced by position embedding is as follows:

[0174] z t =[H t ;Φ(t)] (31)

[0175]

[0176] Among them, [;] represents the concatenation operation, Φ(t) is the time position encoder, d is the dimension of representation, {ω i} i＝1,2,…,d is a learnable parameter, τ t is a timestamp. Then, given the representation sequence z 1:T-1 ={z 1 ,z 2 ,…,z T-1}, using the self-attention mechanism to capture the temporal dependency between position-enhanced representations and output a comprehensive entity representation and comprehensive relationship representation

[0177] Step 15: Use the convolution-based news event model ConvTransE to predict all possible relationships between entities at timestamp T, and use the cross entropy function to calculate the prediction loss of a single sample.

[0178] Given a header entity and tail entity The news event prediction task aims to predict the relationship at the Tth timestamp based on the news events in the historical window [1:T-1]. In this step, the convolution-based news event model ConvTransE is used to predict the probability of different relationships between entities. The calculation formula is as follows:

[0179]

[0180] in, is the probability vector of the relationship, is the set of news events in the historical window [1:T-1], and is a comprehensive entity representation, It is a comprehensive relationship representation matrix, each row corresponds to a comprehensive relationship representation ConvTransE(·) represents a one-dimensional convolutional layer and a fully connected layer.

[0181] Afterwards, based on the prediction results of all relations, the cross entropy loss of a single sample is calculated as the prediction loss The calculation formula is as follows:

[0182]

[0183] Among them, N r Indicates the number of relationships, y i,j Indicates the label of entity relationship j in sample i, y i,j ∈{0,1},p i,j represents the predicted probability of entity relationship j in sample i, which is calculated by formula (33).

[0184] Step 16: The prediction loss of each sample in the batch Time smoothing loss and self-supervised smoothing loss Add up as the prediction loss for all samples in the batch Adjust the learnable parameters of the entire model.

[0185] In this step, the total prediction loss for all samples in the batch is calculated The calculation is as follows:

[0186]

[0187] in, represents the prediction loss of the vth sample in the batch, F represents the number of samples in each batch, represents the time smoothing loss of the v-th sample, represents the self-supervised smoothing loss of the vth sample, and λ and γ represent weights. Afterwards, according to the total prediction loss Adjust the learnable parameters of the entire model.

[0188] Step 17: Repeat steps 4-16 until all batches of the training dataset participate in model training.

[0189] Step 18: Repeat steps 4-17 until the specified number of iterations is reached.

[0190] Step 19: Given the head entity and the tail entity, the processed data samples are input into the trained model to obtain all possible future relationships between the given entities.

[0191] In this step, given the head entity and tail entity, the news events in the historical window are processed and input into the trained model to obtain the representation of the given head entity. and tail entity representation The probability of each relationship occurring between given entities at the next timestamp is calculated using formula (33).

[0192] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A news event prediction method based on heterogeneous evolution event clustering, characterized by: The following steps are involved: An entity graph is constructed based on news events, and a relationship-aware graph convolutional neural network is used to aggregate information on the entity graph to obtain preliminary updated entity and relationship representations. In the event graph construction module, event representations are generated based on the initially updated entity representations and relation representations through a tensor-based relation-aware event encoding module, and an event graph is constructed based on the event representations by treating events as nodes and heterogeneous relations between events as edges. In the event-aware multi-step evolutionary clustering module, a fuzzy clustering algorithm is used to obtain the soft membership matrix of events to event clusters based on the event graph. Event clusters with different historical timestamps are aligned and fused to form an event cluster graph, and a time smoothing loss is constructed based on the distance between event clusters. A self-supervised optimization algorithm is used to optimize the event cluster representation based on the distance and similarity between event clusters on the event cluster graph, and a self-supervised smoothing loss is constructed based on the similarity between event clusters before and after optimization. In the event cluster graph information transfer module, an implicit relation encoder is used to capture the implicit correlation between event clusters and perform sparsification and information aggregation to update the event cluster representation. The event representation is then updated based on the soft membership matrix of the event to the event cluster, thereby updating the entity and relationship representations. The entity representations and relationship representations updated at different timestamps are input into the time-dependent encoder based on the attention mechanism to obtain comprehensive entity representations and relationship representations. All possible future relationships between entities are predicted through the convolution-based news event model. The total prediction loss including prediction loss, temporal smoothing loss and self-supervised smoothing loss is constructed to train the entire network, and the trained news event model is used to predict news events.

2. The news event prediction method based on heterogeneous evolution event clustering according to claim 1 is characterized in that: The method generates an event representation based on the preliminary updated entity representation and relationship representation through a tensor-based relationship-aware event encoding module, including: First, the tensor T is used to encode the element-by-element interaction between entity representation and relation representation. The calculation formula is as follows: Among them, v s,i Represents the interaction v between the head entity s and the relation r s The element v o,i Represents the interaction v between the tail entity o and the relation r o Elements, T ijk Represents the elements of tensor T, subscripts i, j, k represent T ijk The index value in the three directions of the tensor T, h s,j is the jth element of the head entity representation, h o,j is the jth element of the tail entity representation, h r,k is the kth element of the relation; Then, we combine the interaction between the head entity and the relationship v s and the interaction between the tail entity and the relationship v o , we get the event representation ev, the calculation formula is as follows: ev=W s v s +W o v o Among them, W s and W o is a learnable parameter.

3. The news event prediction method based on heterogeneous evolution event clustering according to claim 1 or 2, characterized in that: The event-based representation constructs an event graph by treating events as nodes and heterogeneous relationships between events as edges, including: Based on the event representation ev, the node set of the event graph at timestamp t is represented as Subscript i is the index of the event, using To express events The set of entities contained in in and Represent the head entity and the tail entity respectively; Two types of heterogeneous edges, shared entity edges and adjacent edges, are constructed to describe the relationship between events. They are defined as follows: Shared Entity Edges If two events with indices i and j share at least one entity, there is a shared entity edge between the two events, represented as: adjacent edges If, among two events indexed by i and j, one event is in the k-neighboring set of the other event, then there is a neighboring edge between the two events, which is expressed as: Where kNN(·) represents the k-nearest neighbor algorithm, Indicates equivalence, ∨ indicates or; The two heterogeneous edges together constitute the edge set in the event graph, namely Then, an event graph is constructed based on the node set and edge set. And use the relationship-aware graph convolutional neural network to aggregate information on the event graph.

4. The news event prediction method based on heterogeneous evolution event clustering according to claim 1 is characterized in that: The process of aligning and fusing event clusters with different historical timestamps to form an event cluster graph includes: By aggregating the entities contained in the events in each event cluster, the entity distribution matrix CE of the event cluster at time t is obtained: t , the calculation formula is as follows: WHAT t =(U t ) T M t D -1 Among them, M t represents the event-entity mapping matrix, Indicates M t The i-th element in the event If the entity e is included, its value is 1, otherwise it is 0; D represents a diagonal matrix, each element in D represents the total number of times an entity appears in all events, which is used for normalization; U t represents the soft membership matrix of events to event clusters, Representing an event The set of entities contained in , where the superscript T indicates transposition; Subsequently, the event cluster-aware Hungarian matching algorithm is used to ensure smooth alignment of event clusters with different timestamps. It first constructs the affinity matrix A to quantify the similarity between event clusters with different timestamps. For example, if the event clusters with timestamps t-1 and t are aligned, the calculation formula is as follows: Among them, the element A in the jth row and kth column of A j,k Represents the entity distribution matrix of event cluster j at timestamp t-1 The entity distribution matrix of event cluster k with timestamp t The similarity between them, cos(·) represents the cosine similarity function; After the affinity matrix A is constructed, the event cluster-aware Hungarian matching algorithm is used to find the optimal alignment to maximize the sum of similarities between the aligned event clusters. The calculation formula is as follows: Among them, π represents the permutation function that realizes the smooth alignment of event clusters with different timestamps, π τ (j) represents the index of the event cluster with timestamp τ aligned with the event cluster j with timestamp t, N c represents the total number of event clusters; After aligning the event clusters of historical timestamps, the aligned event clusters are average pooled to obtain the fused event cluster representation. The calculation formula is as follows: Among them, N t Represents the number of timestamps up to and including timestamp t, represents the representation of event cluster j at timestamp t, represents the event cluster of timestamp τ aligned with event cluster j of timestamp t; Then, construct the event cluster graph in, The node set representing the timestamp t represents the event cluster after alignment and fusion. The set of edges representing timestamp t represents the implicit correlation between event clusters.

5. The news event prediction method based on heterogeneous evolution event clustering according to claim 4 is characterized in that: The calculation formula of time smoothing loss is as follows: in, represents the temporal smoothing loss.

6. The news event prediction method based on heterogeneous evolution event clustering according to claim 1 is characterized in that: The method of optimizing the event cluster representation based on the distance and similarity between event clusters on the event cluster graph using a self-supervised optimization algorithm includes: Based on the Euclidean distance between event clusters, a pairwise distance matrix L is constructed, and a distance weight matrix K is constructed. The calculation formula is as follows: Among them, the element L in the jth row and kth column of L j,k Represents the jth event cluster at timestamp t in the event cluster graph constructed after alignment and fusion and the kth event cluster The Euclidean distance between j,k represents the element in the jth row and kth column of K, and α is a hyperparameter used to control the amplitude of the distance weight adjustment; Subsequently, the event cluster similarity matrix F is constructed based on the cosine similarity between event clusters, and the similarity adjustment matrix S is constructed. The calculation formula is as follows: Among them, F j,k represents the element in row j and column k of F, S j,k represents the element in row j and column k of S, and T is a hyperparameter used to control the magnitude of similarity adjustment; The self-supervised optimization of event clusters is achieved by integrating distance weighting and similarity adjustment, and the calculation formula is as follows: in, represents the event cluster after self-supervised optimization, N c Indicates the total number of event clusters.

7. The news event prediction method based on heterogeneous evolution event clustering according to claim 6 is characterized in that: The calculation formula of self-supervised smoothing loss is as follows: in, represents the self-supervised smoothing loss.

8. The news event prediction method based on heterogeneous evolution event clustering according to claim 1 is characterized in that: The convolution-based news event model uses the ConvTransE model.

9. The news event prediction method based on heterogeneous evolution event clustering according to claim 1 is characterized in that: The prediction loss is calculated as follows: in, represents the prediction loss, N r Indicates the number of relationships, y i,j Indicates the label of entity relationship j in sample i, p i,j Represents the predicted probability of entity relationship j in sample i.

10. The news event prediction method based on heterogeneous evolution event clustering according to claim 1 is characterized in that: The total prediction loss is expressed as: in, represents the total prediction loss in a training batch, represents the prediction loss of the vth sample in the batch, F represents the number of samples in each batch, represents the time smoothing loss of the v-th sample, represents the self-supervised smoothing loss of the vth sample, and λ and γ represent weights.

Images