A news event prediction method based on time smoothing constraint deep evolution clustering

Abstract

The application discloses a news event prediction method based on time smoothing constraint deep evolution clustering, comprising the following steps: constructing an entity graph of a news event; using a relation-aware graph convolutional neural network to aggregate information of the entity graph; obtaining a soft membership matrix of the entity to the cluster by using fuzzy clustering in a deep evolution clustering module, and constructing a cluster graph by aligning and fusing clusters of adjacent time stamps; using implicit correlation between cluster pairs in a cluster graph information transmission module to aggregate information to update the representation of the cluster and the representation of the entity and the representation of the relation; combining the updated representation of the entity and the representation of the relation with the initial representation of the entity and the representation of the relation as next time stamp input by using a time residual gate; inputting the updated representation of the entity and the representation of the relation of different time stamps into a time-dependent encoder based on an attention mechanism to obtain comprehensive representation of the entity and the relation; and predicting by a news event model based on convolution. The application can effectively model the time sequence evolution of high-order correlation between entities.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of news event prediction, and particularly relates to a news event prediction method based on time smoothing constraint deep evolution clustering. BACKGROUND

[0002] News event databases, such as the Global Database of Events, Language, and Tone (GDELT) and the Integrated Crisis Early Warning System (ICEWS), contain a large number of time-labeled news events. News events are usually represented in the form of a four-tuple, i.e., (head entity, relation, tail entity, timestamp), which is not only intuitive and expressive, but also accurately captures the core elements of the event and the time point at which it occurred. Among them, entities include countries, regions, citizens, and companies, as participants or affected objects of events, and relations include cooperation, visits, statements, condemnations, and refusals, revealing the behavior or state changes between these entities.

[0003] The news event prediction task maps entities and relations to low-dimensional, dense real-valued vector representations based on historical news events, and then predicts the future possible relations between entities through vector operations. This task not only has great application potential in international event prediction, social risk assessment, and public safety management, but also is an important embodiment of the deep integration of artificial intelligence and big data technology. However, news event prediction faces many challenges, including but not limited to the massive, heterogeneous, and time-dependent nature of the data. In particular, how to effectively capture the high-order correlation between entities and its evolution over time has become a hot and difficult point in current research.

[0004] To address these challenges, researchers have turned to deep neural networks, particularly graph neural networks (GNNs) and recurrent neural networks (RNNs) and their variants. Graph neural networks are good at processing graph-structured data and can naturally capture pairwise correlations between entities, enabling relationship reasoning and knowledge representation through information aggregation and transmission between nodes. Recurrent neural networks, on the other hand, are known for their strong modeling capabilities for time series data, capable of capturing temporal dependencies between events. Deep neural network-based news event prediction methods use graph neural networks and recurrent neural networks and their variants to model graph structure information and temporal dependencies in news events, respectively, and these methods have achieved good results in event prediction tasks. In these methods, the graph is constructed based on quadruples, and graph neural networks are used to capture pairwise correlations between entities. Although graph neural network and recurrent neural network-based methods have made significant progress, existing models still have limitations in capturing high-order correlations between entities and their temporal evolution. High-order correlations refer to complex associations between three or more entities, which often contain deeper social logic and dynamic rules. To break this bottleneck, recent methods attempt to introduce derived structures such as entity groups, communities, and hypergraphs to more finely capture the relationships between entities. However, these methods cannot capture the temporal evolution of high-order correlations between entities. SUMMARY

[0005] In view of the above, the purpose of the present application is to provide a news event prediction method based on time smoothing constraint deep evolution clustering, which constructs an entity graph based on news events occurring at each timestamp, uses a deep evolution clustering module to construct clusters, aligns and fuses clusters of adjacent timestamps to construct a cluster graph, and then constructs a time smoothing constraint, and introduces a cluster graph information transmission module for representation learning, thereby effectively modeling the temporal evolution of high-order correlations between entities, and having broad application prospects in international event prediction, social governance, etc.

[0006] To achieve the above invention purpose, the technical solutions provided by the present application are as follows:

[0007] The news event prediction method based on time smoothing constraint deep evolution clustering provided by the embodiment of the present application comprises the following steps:

[0008] In the constructed training data set, an entity graph is constructed based on news events occurring at each timestamp, with entities as nodes and relationships between entities as edges;

[0009] A relation-aware graph convolutional neural network is used to aggregate information on the entity graph to obtain preliminary updated entity representations and relationship representations;

[0010] In the deep evolution clustering module, the soft membership matrix of entities to clusters and clusters of different timestamps are obtained based on the preliminary updated entity representation and the fuzzy clustering, the cluster graph is constructed by aligning and merging the clusters of adjacent timestamps, and the time smoothing loss is constructed according to the cosine distance between the clusters of adjacent timestamps;

[0011] In the cluster graph information transmission module, the implicit correlation between the cluster pairs in the cluster graph is captured by using the implicit correlation encoder, and then the representation of the cluster is updated by aggregating information on the cluster graph, and then the preliminary updated entity representation and relationship representation are updated again according to the soft membership matrix of entities to clusters;

[0012] The entity representation and relationship representation updated again at the current timestamp are combined with the initial entity representation and relationship representation at the current timestamp by using the time residual gate, and are used as the input of the next timestamp;

[0013] The entity representation and relationship representation updated again at different timestamps are input into the time-dependent encoder based on the attention mechanism to obtain the comprehensive entity representation and relationship representation;

[0014] Based on the comprehensive entity representation and relationship representation, all relationships that may occur between entities in the future are predicted by the news event model based on convolution, the total prediction loss including the prediction loss and the time smoothing loss is constructed, the entire network is trained based on the training data set and the total prediction loss, and the news event prediction is performed by using the trained news event model.

[0015] Specifically, the entity graph is constructed based on the news events occurring at each timestamp in the constructed training data set, including:

[0016] The news event data is preprocessed, and the obtained news events are represented in the form of a quadruple (s, r, o, t), wherein s represents the head entity, r represents the relationship, o represents the tail entity, and t represents the timestamp;

[0017] The preprocessed news event data is divided into a training data set by using a sliding time window with a size of T, wherein each sample contains news events occurring at T timestamps in the news event data;

[0018] All news events in the training data set are constructed into a global static graph, wherein the nodes represent entities, and the edges represent the relationships between entities, and the global clusters are obtained based on the global static graph by using spectral clustering;

[0019] The training data set is divided into N batches for batch training, and the learnable entity representation e i and the learnable relationship representation r ii represents the index of the entity representation or the relationship representation;

[0020] A batch of training samples are selected, and an entity graph is constructed according to a news event occurring at each timestamp wherein, represents a node set of the t timestamp, representing an entity; represents an edge set of the t timestamp, representing the relationship between entities.

[0021] Specifically, the graph convolutional neural network with relationship awareness performs information aggregation on the entity graph to obtain the preliminary updated entity representation and relationship representation, including:

[0022] The calculation formula of the graph convolutional neural network with relationship awareness for information aggregation on the entity graph is as follows:

[0023]

[0024] wherein, represents the preliminary updated tail entity representation, since the tail entity in the news event can also be regarded as the head entity of another event, the head entity representation is also regarded as being synchronously updated, and r t,l respectively represent the representation of the head entity s, the tail entity o and the relationship r at the t timestamp at the l layer, represents a news event set at the t timestamp, W1 and W2 represent learnable parameters, d o represents the in-degree of the tail entity, and RReLU(·) represents an activation function;

[0025] the preliminary updated relationship representation r of the t timestamp t obtained from the entity representation related to r and the representation r of r at the previous timestamp t-1 , and the calculation formula is as follows:

[0026]

[0027] wherein, [;] represents a concatenation operation, and pooling(·) represents an average pooling operation, represents the preliminary updated entity representation related to r at the t timestamp, wherein the entity set of the head entity or the tail entity x represents the head entity in (x, r, o, t), and x represents the tail entity in (s, r, x, t).

[0028] Specifically, the fuzzy clustering based on the preliminary updated entity representation in the deep evolution clustering module obtains the soft membership matrix of the entity to the cluster and the cluster at different timestamps, including:

[0029] In the deep evolutionary clustering module, the soft membership matrix U of an entity to a cluster is first obtained by optimizing the following objective function J, and the calculation formula is as follows:

[0030]

[0031] Where, N e and N c Let i represent the number of entities and clusters, j represent the index of the entity, and j represent the index of the cluster. The element in the i-th row and j-th column of the soft membership matrix U is... Entity representation of the initial update of timestamp t For cluster centroid The membership degree, the superscript m indicates the fuzzy smoothing hyperparameter, <·> indicates the dot product operation, and ‖·‖ indicates the magnitude of the vector;

[0032] Then, the overlapping clusters of timestamp t are obtained, calculated as follows:

[0033]

[0034] in, The representation of the j-th cluster at timestamp t is obtained by weighting the entity representation and the entity's membership degree to the cluster. Indicates the cluster centroid Cluster centroids with index k other than k This represents the set of nodes in the entity graph.

[0035] Specifically, the step of aligning and fusing clusters with adjacent timestamps to construct a cluster graph with clusters as nodes and implicit correlations between clusters as edges includes:

[0036] In the deep evolutionary clustering module, a cluster-aware Hungarian matching algorithm is used to ensure smooth alignment of clusters with adjacent timestamps. It first constructs an affinity matrix A to quantify the similarity between clusters with consecutive timestamps, calculated as follows:

[0037]

[0038] Where, element a in row j and column k of A j,k This represents the j-th cluster of timestamp t-1. The kth cluster of timestamp t The similarity between them, cos(·) represents the cosine similarity function;

[0039] After the affinity matrix A is constructed, the cluster-aware Hungarian matching algorithm is used to find the optimal alignment, which maximizes the sum of similarities between aligned clusters. The calculation formula is as follows:

[0040]

[0041] Where π represents the permutation function for achieving smooth cluster alignment of adjacent timestamps, and π(j) represents the index of the cluster at timet t that is aligned with the j-th cluster at timet-1.

[0042] After aligning clusters with adjacent timestamps, the aligned clusters are weighted and fused to obtain the fused cluster representation. The calculation formula is as follows:

[0043]

[0044] Where β is the weighted fusion hyperparameter, representing the relative contribution of cluster representations with adjacent timestamps to the fused cluster representation;

[0045] Subsequently, a cluster graph is constructed. in, The set of nodes representing timestamp t represents the aligned and merged cluster. The edge set representing timestamp t represents the implicit correlation between clusters after alignment and fusion.

[0046] Specifically, the formula for calculating the time smoothing loss is as follows:

[0047]

[0048] in, This represents the time smoothing loss.

[0049] Specifically, in the cluster graph information transmission module, after capturing the implicit correlations between cluster pairs in the cluster graph using an implicit correlation encoder, information aggregation is performed on the cluster graph to update the cluster representation. Then, based on the soft membership matrix of the entity to the cluster, the initially updated entity and relation representations are updated again, including:

[0050] The implicit correlation between cluster pairs is captured using the implicit correlation encoder in the cluster graph information transmission module. This includes the implicit correlation between cluster pairs at timestamp t, specifically the correlation between the i-th and j-th cluster pairs. The calculation formula is as follows:

[0051]

[0052] in, ReLU represents the transformation function, implemented by a multilayer perceptron, and ReLU(·) represents the activation function.

[0053] The strength of implicit correlation is quantified to obtain The calculation formula is as follows:

[0054]

[0055] Where σ(·) represents the sigmoid function and Conv(·) represents the convolution operation;

[0056] Based on the global clusters obtained from the global static graph, the similarity between the global clusters is used to enhance the strength of the implicit correlation between the cluster pairs, and the calculation formula is as follows:

[0057]

[0058]

[0059] wherein, denotes the global cluster aligned with the i-th cluster and the j-th cluster at the t timestamp, denotes the similarity between the i-th global cluster and the i-th global cluster at the t timestamp, denotes the implicit correlation strength of the t timestamp after global cluster enhancement, which integrates the structure information from the global graph;

[0060] The calculation formula of the information transmission process of the cluster graph is as follows:

[0061]

[0062] wherein, denotes the t timestamp and the i-th cluster c i corresponding vector representation;

[0063] Then, the cluster representation and the entity representation are updated, and the calculation formula is as follows:

[0064]

[0065] wherein, denotes the updated cluster representation of the i-th cluster at the t timestamp, denotes the element in the i-th row and the j-th column of the soft membership matrix, denotes the updated cluster representation of the j-th cluster of the N c clusters at the t timestamp, denotes the i-th entity representation updated again at the t timestamp, and the relationship representation updated again is obtained according to the entity representation updated again.

[0066] Specifically, the news event model based on convolution adopts the ConvTransE model.

[0067] Specifically, the calculation formula of the prediction loss is as follows:

[0068]

[0069] wherein, denotes the prediction loss, N r denotes the number of relationships, y i,j denotes the label of the entity in the event i to have the relationship j, p i,jdenotes the prediction probability of the entity in the event i occurring the relationship j.

[0070] Specifically, the entire network is trained based on the training data set and the total prediction loss, including:

[0071] The training data set is divided into batches, and the prediction loss of each sample in the batch and the time smoothing loss are added as the total prediction loss of all samples in the batch The learnable parameters in the entire network are adjusted, and the total prediction loss of all samples in the batch is calculated as follows:

[0072]

[0073] Wherein, denotes the prediction loss of the vth sample in the batch, F denotes the number of samples in each batch, denotes the time smoothing loss of the vth sample.

[0074] Compared with the prior art, the present application has at least the following beneficial effects:

[0075] (1) The present application introduces a deep evolution clustering module, uses fuzzy clustering to obtain a soft membership matrix of entity pairs, and uses a cluster-aware unsupervised alignment mechanism to align and fuse clusters of adjacent timestamps to construct a cluster graph, and constructs a time smoothing constraint between clusters of adjacent timestamps to model the temporal evolution of high-order correlation between entities.

[0076] (2) The present application introduces a cluster graph information transmission module, models the cluster graph as a fully connected graph, infers the implicit correlation between clusters in an end-to-end manner, and aggregates and updates the information according to the cluster graph, so as to use the temporal evolution of high-order correlation between entities for representation learning, thereby realizing accurate modeling of the temporal evolution of high-order correlation between entities. BRIEF DESCRIPTION OF DRAWINGS

[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0078] Figure 1 is a whole flowchart of a news event prediction method based on time smoothing constraint deep evolution clustering provided by the embodiment of the present application;

[0079] Figure 2This is an overall framework diagram of a news event prediction method based on time-smoothed constrained deep evolutionary clustering provided by an embodiment of the present invention. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0081] The inventive concept of this invention is as follows: Addressing the problem that existing news event prediction tasks cannot capture the temporal evolution of high-order correlations between entities, this invention provides a news event prediction method based on time-smoothing constraint deep evolutionary clustering. It constructs an entity graph based on news events occurring at each timestamp, uses a deep evolutionary clustering module to construct clusters, aligns and merges clusters of adjacent timestamps to construct a cluster graph, then constructs time-smoothing constraints, and introduces a cluster graph information transfer module for representation learning to model the temporal evolution of high-order correlations between entities.

[0082] Figure 1 This is an overall flowchart of a news event prediction method based on time-smoothed constrained deep evolutionary clustering provided by an embodiment of the present invention. Figure 2 This is an overall framework diagram of a news event prediction method based on time-smoothed constrained deep evolutionary clustering provided by an embodiment of the present invention. Figure 1 and Figure 2 As shown in the embodiment, a news event prediction method based on time-smoothed constrained deep evolutionary clustering is provided, including the following steps:

[0083] Step 1: Preprocess the news event data and extract the news events. Divide the processed data into a training dataset using a sliding time window of size T, where each sample contains news events occurring at T timestamps from the news event data.

[0084] The news event data contains a large number of time-stamped news events, which are then extracted. News events are represented as quadruples (s, r, o, t), where s represents the head entity, r represents the relation, o represents the tail entity, and t represents the timestamp. ε represents the entity set, mainly including entities such as countries, regions, citizens, and companies, and s, o ∈ ε. The set of representative relationships mainly includes those involving cooperation, visits, statements, condemnation, and rejection. Represents a set of timestamps, and

[0085] The timestamps of the quadruplets represent the times when the news events occurred. The quadruplets are arranged in chronological order, and then the processed data is divided using a sliding time window of size T to obtain the training dataset. This represents the set of news events occurring at timestamp t, where each sample contains a set of news events occurring at T timestamps in the news data.

[0086] Step 2: Construct a global static graph G from all news events in the training dataset. global In this model, nodes represent entities, edges represent relationships between entities, and spectral clustering is used to obtain global clusters.

[0087] In this step, all news events in the training dataset are constructed into a global static graph, and global clusters are obtained using spectral clustering. The calculation formula is as follows:

[0088]

[0089] in, Let represent the i-th global cluster, and SC(·) represent the spectral clustering algorithm.

[0090] Step 3: Randomly initialize the learnable entity representation e based on the in-degree of the nodes in the global static graph. i Randomly initialized learnable relational representation r i , where i represents the index of the entity representation or relation representation.

[0091] Step 4: Divide the training dataset into batches of a fixed batch size, with a total of N batches.

[0092] Step 5: Select a batch of training samples with index k sequentially from the training dataset, where k∈{1,…,N}. Repeat steps 6-15 for each training sample in this batch.

[0093] Step 6: Construct an entity graph based on the news events that occurred at each timestamp. in, The set of nodes representing timestamp t represents an entity; The edge set representing timestamp t represents the relationship between entities.

[0094] Step 7: For timestamps t = 1, 2, ..., T-1, iteratively execute steps 8-12.

[0095] Step 8: Use a relation-aware graph convolutional neural network to aggregate information on the entity graph.

[0096] In this step, the calculation formula for information aggregation on the entity graph using a relation-aware graph convolutional neural network is as follows:

[0097]

[0098] in, This indicates an initial update to the tail entity representation. Since tail entities in news events can also serve as head entities for another event, this is also considered a synchronous initial update to the head entity representation. and r t,l Let represent the header entity s, the tail entity o, and the relation r at time stamp t, respectively, at level l. Let d represent the set of news events with timestamp t, W1 and W2 represent learnable parameters, and d o `RReLU(·)` represents the in-degree of the tail entity, and `RReLU(·)` represents the activation function. In the following sections, for brevity and to avoid ambiguity, the superscript `l` of variables will be omitted. The relational representation of the initial update of the timestamp `t` is `r`. t The entity associated with r and the representation of r at the previous timestamp. t-1 The calculation formula is as follows:

[0099]

[0100] Where [;] denotes concatenation operation, and pooling(·) denotes average pooling operation. This represents the entity representation of the initial update associated with time stamp t and r, where the entity is either the head entity or the tail entity. In (x,r,o,t), x represents the head entity, and in (s,r,x,t), x represents the tail entity.

[0101] Step 9: Use fuzzy clustering in the deep evolutionary clustering module to obtain the soft membership matrix U of entity pairs and the clusters of timestamp t.

[0102] In this step, the soft membership matrix U of the entity to the cluster is first obtained by optimizing the following objective function J, and the calculation formula is as follows:

[0103]

[0104] Where, N e and N c Let i represent the number of entities and clusters, j represent the index of the entity, and j represent the index of the cluster. The element in the i-th row and j-th column of the soft membership matrix U is... Entity representation of the initial update of timestamp t For cluster centroid The membership degree, the superscript m indicates the fuzzy smoothing hyperparameter, the value of m is set to be greater than 1, <·> indicates the dot product operation, and ‖·‖ indicates the magnitude of the vector.

[0105] Then, the overlapping clusters of timestamp t are obtained, calculated as follows:

[0106]

[0107] in, The representation of cluster j at timestamp t is obtained by weighting the entity representation and the entity's membership degree to the cluster. Indicates the cluster centroid The centroid of the cluster with index k other than k.

[0108] Step 10: Use a cluster-aware unsupervised alignment mechanism to align and merge clusters with adjacent timestamps to construct a cluster graph. in, The set of nodes representing timestamp t represents a cluster; E c t Let t represent the edge set of timestamp t, which represents the implicit correlation between clusters, and derive the time smoothing loss based on the cosine distance between clusters with adjacent timestamps. As a time smoothing constraint.

[0109] In this step, a cluster-aware Hungarian matching algorithm is used to ensure smooth alignment of clusters with adjacent timestamps. It first constructs an affinity matrix A to quantify the similarity between clusters with consecutive timestamps, calculated as follows:

[0110]

[0111] Where, element a in row j and column k of A j,k This represents the j-th cluster of timestamp t-1. The kth cluster of timestamp t The similarity between them is expressed by cos(·), which represents the cosine similarity function.

[0112] After the affinity matrix A is constructed, the cluster-aware Hungarian matching algorithm is used to find the optimal alignment, maximizing the sum of similarities between aligned clusters. The process of finding the optimal alignment can be viewed as an optimization problem, and the calculation formula is as follows:

[0113]

[0114] Where π represents the permutation function for achieving smooth alignment of clusters with adjacent timestamps, and π(j) represents the index of the cluster at timestamp t that is aligned with the j-th cluster at timestamp t-1.

[0115] After aligning clusters with adjacent timestamps, the aligned clusters are weighted and fused to obtain the fused cluster representation. The calculation formula is as follows:

[0116]

[0117] Where β is the weighted fusion hyperparameter, representing the relative contribution of cluster representations with adjacent timestamps to the fused cluster representation.

[0118] Subsequently, a cluster graph is constructed. in, The set of nodes representing timestamp t represents the cluster after alignment and merging; Let represent the edge set for timestamp t, which represents the implicit correlation between clusters after alignment and fusion. Furthermore, the temporal smoothing loss is derived from the cosine distance between clusters with adjacent timestamps. As a time smoothing constraint, the calculation formula is as follows:

[0119]

[0120] Step 11: The implicit correlation encoder in the cluster graph information transfer module is used to capture the implicit correlation between cluster pairs, and the strength of the implicit correlation between cluster pairs is enhanced by utilizing the global cluster similarity. A graph convolutional neural network is used to aggregate information on the cluster graph to update the cluster representation, and the entity representation is updated according to the soft membership matrix U of the entity pairs to the clusters. And update the relation representation based on the updated entity representation.

[0121] In this step, the implicit correlation between cluster pairs is captured using the implicit correlation encoder in the cluster graph information transfer module. The implicit correlation between cluster pairs at timestamp t (i-th and j-th clusters) is then defined. The calculation formula is as follows:

[0122]

[0123] in, ReLU represents the transformation function, implemented by a multilayer perceptron, and ReLU(·) represents the activation function.

[0124] Furthermore, the strength of implicit correlations is quantified to obtain... The calculation formula is as follows:

[0125]

[0126] Where σ(·) represents the sigmoid function, which ensures that the obtained implicit correlation strength value is between 0 and 1, and Conv(·) represents the convolution operation.

[0127] Higher interaction frequency over a longer time span indicates stronger implicit correlation. Therefore, based on global clusters obtained from a global static graph, the similarity between global clusters is used to enhance the strength of implicit correlations between cluster pairs, calculated as follows:

[0128]

[0129] in, The similarity matrix represents the global clusters aligned with the i-th and j-th clusters at timet t. element values ​​in Let represent the similarity between the i-th global cluster and the i-th global cluster at timestamp t. This represents the implicit correlation strength of the t timestamp after global cluster enhancement, which integrates structural information from the global graph.

[0130] The calculation formula for the information transfer process in a cluster graph is as follows:

[0131]

[0132] in, Represents the timestamp t and the i-th cluster c i The corresponding vector representation.

[0133] Subsequently, the cluster representation and entity representation are updated, calculated as follows:

[0134]

[0135] in, This represents the updated cluster representation of the i-th cluster at timestamp t. This represents the element in the i-th row and j-th column of the soft membership matrix. Indicates that there are N timestamps in total. c The updated cluster representation of the j-th cluster in a set of clusters. This represents the entity representation after the i-th update at timestamp t, and the updated relation representation is obtained based on the updated entity representation. For brevity, the indices i and j of the variables will be omitted in the following content.

[0136] Step 12: Use the time residual gate to combine the updated entity and relation representations output at timet t with the entity and relation representations input to the model at timet t to generate the model input at timet t+1.

[0137] In this step, the updated entity and relation representations output at timetamp t are combined with the representation of the input model at timetamp t using a time residual gate. The calculation formula is as follows:

[0138]

[0139] in, This represents the dot product operation. H is the updated entity or relation representation output by timestamp t. t-1 It is the updated entity or relation representation output at timestamp t-1, that is, the representation of the input model at timestamp t, X t This is a time residual gate, which determines the proportion of information to be retained. The calculation formula is as follows:

[0140] X t =σ(W3H t-1 +b) (19)

[0141] Where σ is the sigmoid function, and W3 and b are learnable parameters.

[0142] Step 13: Input the entity and relation representations updated with different timestamps into an attention-based temporal dependency encoder to obtain a comprehensive entity and relation representation.

[0143] In this step, an attention-based temporal dependency encoder is introduced to model the temporal dependencies between representations. The computation formula for the location-embedded enhanced representation is as follows:

[0144] z t =[H t ;Φ(t)] (20)

[0145]

[0146] Where [;] denotes a concatenation operation, Φ(t) is the time-position encoder, d is the dimension represented, and {ω i} i＝1,2,…,d It is a learnable parameter, τ t It is a timestamp. Subsequently, given a representation sequence z... 1:T-1 ={z 1 ,z 2 ,…,z T-1 It utilizes a self-attention mechanism to capture the temporal dependencies between location-enhanced representations, outputting a comprehensive entity representation. The relationship between and synthesis

[0147] Step 14: Use the convolution-based news event model ConvTransE to predict all possible relationships between entities at timetamp T, and use the cross-entropy function to calculate the prediction loss for a single sample.

[0148] Given head entity Tail-end entity The news event prediction task aims to predict the relationship at the T-th timestamp based on news events in a historical window [1:T-1]. In this step, the convolutional news event model ConvTransE is used to predict the probability of different relationships occurring between entities, calculated as follows:

[0149]

[0150] in, It is a probability vector of the relationship. It is a collection of news events in the history window [1:T-1]. It is a comprehensive entity representation, H r It is a comprehensive relational representation matrix, where each row corresponds to a comprehensive relational representation. ConvTransE(·) represents a one-dimensional convolutional layer and a fully connected layer.

[0151] Next, based on the prediction results of all relationships, the cross-entropy loss of a single sample is calculated using the following formula:

[0152]

[0153] in N represents the predicted loss. r y represents the number of relations. i,j The label y represents the relationship j between entities in event i. i,j ∈{0,1}, p i,j The predicted probability of entity j in event i is calculated by formula (22).

[0154] Step 15: Calculate the prediction loss for each sample in the batch. and time smoothing loss The sum is the total prediction loss for all samples in this batch. Adjust the learnable parameters throughout the network.

[0155] In this step, the prediction loss for all samples in the batch is calculated as follows:

[0156]

[0157] in, Let F represent the prediction loss for the v-th sample in the batch, and let F represent the number of samples in each batch. Let represent the time smoothing loss for the v-th sample. Then, based on the prediction loss... Adjust the learnable parameters throughout the network.

[0158] Step 16: Repeat steps 5-15 until all batches of the training dataset have been used in model training.

[0159] Step 17: Repeat steps 5-16 until the specified number of iterations is reached.

[0160] Step 18: Input the processed data sample into the pre-trained convolution-based news event model ConvTransE to obtain all possible future relationships between given entities.

[0161] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A news event prediction method based on time-smoothed constrained deep evolutionary clustering, characterized in that, Includes the following steps: In the constructed training dataset, an entity graph is constructed based on the news events that occur at each timestamp. Entities are the participants or affected objects of the event, and the relationships reveal the behavior or state changes between these entities. Using a relation-aware graph convolutional neural network, information is aggregated on the entity graph to obtain a preliminary updated entity representation and relation representation; In the deep evolution clustering module, based on the initially updated entity representation, fuzzy clustering is used to obtain the soft membership matrix of the entity to the cluster and the clusters with different timestamps. After aligning and fusing the clusters with adjacent timestamps, a cluster graph is constructed with clusters as nodes and implicit correlations between clusters as edges. The temporal smoothing loss is constructed based on the cosine distance between clusters with adjacent timestamps. In the cluster graph information transmission module, the implicit correlation encoder is used to capture the implicit correlation between cluster pairs in the cluster graph. Then, information aggregation is performed on the cluster graph to update the cluster representation. Finally, the entity representation and relation representation that were initially updated are updated again based on the soft membership matrix of the entity to the cluster. The entity and relation representations updated at the current timestamp are combined with the initial entity and relation representations at the current timestamp using the time residual gate, and used as input for the next timestamp; The entity and relation representations, updated again with different timestamps, are input into an attention-based temporal dependency encoder to obtain a comprehensive entity and relation representation. Based on comprehensive entity and relation representations, this method predicts all possible future relationships between entities using a convolutional news event model. It constructs a total prediction loss that includes prediction loss and temporal smoothing loss, trains the entire network on the training dataset and the total prediction loss, and uses the trained news event model to predict news events.

2. The news event prediction method based on time-smoothed constrained deep evolutionary clustering according to claim 1, characterized in that, The construction of an entity graph in the constructed training dataset, based on news events occurring at each timestamp, with entities as nodes and relationships between entities as edges, includes: The news event data is preprocessed, and the extracted news events are represented in the form of a quadruple (s,r,o,t), where s represents the head entity, r represents the relation, o represents the tail entity, and t represents the timestamp. The preprocessed news event data is divided into training datasets using a sliding time window of size T, where each sample contains news events that occurred at T timestamps in the news event data; All news events in the training dataset are constructed into a global static graph, where nodes represent entities and edges represent relationships between entities. Global clusters are obtained based on the global static graph using spectral clustering. The training dataset is divided into N batches for batch training, and the learnable entity representation e is randomly initialized. i The learnable relation is represented by r. i , where i represents the index of the entity representation or relation representation; Select a batch of training samples and construct an entity graph based on the news events that occur at each timestamp. in, The set of nodes representing timestamp t represents an entity; The edge set representing timestamp t represents the relationship between entities.

3. The news event prediction method based on time-smoothed constrained deep evolutionary clustering according to claim 1, characterized in that, The relation-aware graph convolutional neural network is used to aggregate information on the entity graph to obtain a preliminary updated entity representation and relation representation, including: The calculation formula for information aggregation on entity graphs using relation-aware graph convolutional neural networks is as follows: in, This indicates an initial update to the tail entity representation. Since tail entities in news events can also serve as head entities for another event, this is also considered a synchronous initial update to the head entity representation. and r t,l Let represent the header entity s, the tail entity o, and the relation r at time stamp t, respectively, at level l. Let d represent the set of news events with timestamp t, W1 and W2 represent learnable parameters, and d o RReLU(·) represents the in-degree of the tail entity, and RReLU(·) represents the activation function. The relational representation of the initial update of timestamp t. t The entity associated with r and the representation of r at the previous timestamp. t-1 The calculation formula is as follows: Where [;] denotes concatenation operation, and pooling(·) denotes average pooling operation. This represents the entity representation of the initial update associated with time stamp t and r, where the entity is either the head entity or the tail entity. In (x,r,o,t), x represents the head entity, and in (s,r,x,t), x represents the tail entity.

4. The news event prediction method based on time-smoothed constrained deep evolutionary clustering according to claim 1, characterized in that, The process of obtaining the soft membership matrix of entities to clusters and clusters with different timestamps for each entity using fuzzy clustering based on the initially updated entity representation in the deep evolutionary clustering module includes: In the deep evolutionary clustering module, the soft membership matrix U of an entity to a cluster is first obtained by optimizing the following objective function J, and the calculation formula is as follows: Where, N e and N c Let i represent the number of entities and clusters, j represent the index of the entity, and j represent the index of the cluster. The element in the i-th row and j-th column of the soft membership matrix U is... Entity representation of the initial update of timestamp t For cluster centroid The membership degree, the superscript m indicates the fuzzy smoothing hyperparameter, <·> indicates the dot product operation, and ‖·‖ indicates the magnitude of the vector; Then, the overlapping clusters of timestamp t are obtained, calculated as follows: in, The representation of the j-th cluster at timestamp t is obtained by weighting the entity representation and the entity's membership degree to the cluster. Indicates the cluster centroid Cluster centroids with index k other than k This represents the set of nodes in the entity graph.

5. The news event prediction method based on time-smoothed constrained deep evolutionary clustering according to claim 1, characterized in that, The process of aligning and fusing clusters with adjacent timestamps to construct a cluster graph with clusters as nodes and implicit correlations between clusters as edges includes: In the deep evolutionary clustering module, a cluster-aware Hungarian matching algorithm is used to ensure smooth alignment of clusters with adjacent timestamps. It first constructs an affinity matrix A to quantify the similarity between clusters with consecutive timestamps, calculated as follows: Where, element a in row j and column k of A j,k This represents the j-th cluster of timestamp t-1. The kth cluster of timestamp t The similarity between them, cos(·) represents the cosine similarity function; After the affinity matrix A is constructed, the cluster-aware Hungarian matching algorithm is used to find the optimal alignment, which maximizes the sum of similarities between aligned clusters. The calculation formula is as follows: Where π represents the permutation function for achieving smooth cluster alignment of adjacent timestamps, and π(j) represents the index of the cluster at timet t that is aligned with the j-th cluster at timet-1. After aligning clusters with adjacent timestamps, the aligned clusters are weighted and fused to obtain the fused cluster representation. The calculation formula is as follows: Where β is the weighted fusion hyperparameter, representing the relative contribution of cluster representations with adjacent timestamps to the fused cluster representation; Subsequently, a cluster graph is constructed. in, The set of nodes representing timestamp t represents the aligned and merged cluster. The edge set representing timestamp t represents the implicit correlation between clusters after alignment and fusion.

6. The news event prediction method based on time-smoothed constrained deep evolutionary clustering according to claim 5, characterized in that, The formula for calculating the time smoothing loss is as follows: in, This represents the time smoothing loss.

7. The news event prediction method based on time-smoothed constrained deep evolutionary clustering according to claim 2, characterized in that, The process of capturing implicit correlations between cluster pairs in the cluster graph using an implicit correlation encoder in the cluster graph information transmission module, aggregating information on the cluster graph to update the cluster representation, and then updating the initially updated entity and relation representations again based on the soft membership matrix of the entity to the cluster, includes: The implicit correlation between cluster pairs is captured using the implicit correlation encoder in the cluster graph information transmission module. This includes the implicit correlation between cluster pairs at timestamp t, specifically the correlation between the i-th and j-th cluster pairs. The calculation formula is as follows: in, ReLU represents the transformation function, implemented by a multilayer perceptron, and ReLU(·) represents the activation function. The strength of implicit correlation is quantified to obtain The calculation formula is as follows: Where σ(·) represents the sigmoid function and Conv(·) represents the convolution operation; Based on the global clusters obtained from the global static graph, the strength of the implicit correlation between cluster pairs is enhanced by utilizing the similarity between global clusters. The calculation formula is as follows: in, and This represents the global cluster aligned with the i-th and j-th clusters at timestamp t. Let represent the similarity between the i-th global cluster and the i-th global cluster at timestamp t. This represents the implicit correlation strength of the t timestamp after global cluster enhancement, which integrates structural information from the global graph; The calculation formula for the information transfer process in a cluster graph is as follows: in, Represents the timestamp t and the i-th cluster c i The corresponding vector representation; Subsequently, the cluster representation and entity representation are updated, calculated as follows: in, This represents the updated cluster representation of the i-th cluster at timestamp t. This represents the element in the i-th row and j-th column of the soft membership matrix. Indicates that there are N timestamps in total. c The updated cluster representation of the j-th cluster in a set of clusters. Let t represent the entity representation after the i-th time stamp is updated, and obtain the updated relation representation based on the updated entity representation.

8. The news event prediction method based on time-smoothed constrained deep evolutionary clustering according to claim 1, characterized in that, The convolution-based news event model uses the ConvTransE model.

9. The news event prediction method based on time-smoothed constrained deep evolutionary clustering according to claim 1, characterized in that, The formula for calculating the predicted loss is as follows: in, N represents the predicted loss. r y represents the number of relations. i,j p represents the label indicating the relationship j between entities in event i. i,j This represents the predicted probability that entities in event i will have a relationship with j.

10. The news event prediction method based on time-smoothed constrained deep evolutionary clustering according to claim 1, characterized in that, The training of the entire network based on the training dataset and the total prediction loss includes: Divide the training dataset into batches and calculate the prediction loss for each sample in each batch. and time smoothing loss The sum is the total prediction loss for all samples in this batch. Adjusting the learnable parameters across the entire network, the total prediction loss for all samples in the batch. The calculation method is as follows: in, Let F represent the prediction loss for the v-th sample in the batch, and let F represent the number of samples in each batch. This represents the temporal smoothing loss for the v-th sample.

Images