News event prediction method based on time smooth constraint depth evolution clustering

Abstract

The invention discloses a news event prediction method based on time smooth constraint depth evolution clustering. The method comprises the following steps: constructing an entity map of a news event; performing information aggregation on the entity graph by using a relationship-aware graph convolutional neural network; in the deep evolution clustering module, fuzzy clustering is utilized to obtain a soft membership matrix of an entity to clusters, and the clusters of adjacent timestamps are aligned and fused to construct a cluster graph; in the cluster graph information transmission module, information aggregation is carried out by using implicit correlation between cluster pairs so as to update cluster representation, entity representation and relation representation; combining the updated entity representation and relationship representation with the initial entity representation and relationship representation by using a time residual gate to serve as a next timestamp input; inputting the entity representation and the relation representation updated by different timestamps into a time-dependent encoder based on an attention mechanism to obtain a comprehensive entity representation and a relation representation; and predicting through the news event model based on convolution. According to the method, time sequence evolution of high-order correlation between entities can be effectively modeled.

Description

Technical Field

[0001] The present invention belongs to the technical field of news event prediction, and in particular relates to a news event prediction method based on time smoothing constrained deep evolutionary clustering. Background Art

[0002] News event databases, such as the Global Database of Events, Language, and Tone (GDELT) and the Integrated Crisis Early Warning System (ICEWS), contain a large number of news events with time stamps. News events are usually represented in the form of a quadruple, namely (head entity, relationship, tail entity, timestamp). This representation method is not only intuitive and expressive, but also can accurately capture the core elements of the event and the time point of its occurrence. Among them, entities include countries, regions, citizens, and companies as participants or affected objects of the event, and relationships include cooperation, visits, statements, condemnation, and rejection, which reveal the behavior or state changes between these entities.

[0003] The news event prediction task is based on historical news events, mapping entities and relationships into low-dimensional, dense real-valued vector representations, and then predicting possible future relationships between entities through vector operations. This task not only shows great application potential in the fields of international event prediction, social risk assessment, and public safety management, but is also an important manifestation of the deep integration of artificial intelligence and big data technology. However, news event prediction faces many challenges, including but not limited to the massiveness, heterogeneity, and time-series dependency of data. In particular, how to effectively capture the high-order correlations between entities and their evolution over time has become a hot topic and difficulty in current research.

[0004] To address these challenges, researchers have turned their attention to deep neural networks, especially graph neural networks (GNNs) and recurrent neural networks (RNNs) and their variants. Graph neural networks are good at processing graph-structured data and can naturally capture pairwise correlations between entities. They can achieve relational reasoning and knowledge representation through information aggregation and transmission between nodes. RNNs are known for their powerful modeling capabilities for time series data and can capture the temporal dependencies between events. News event prediction methods based on deep neural networks use graph neural networks and recurrent neural networks and their variants to model graph structure information and temporal dependencies in news events respectively. These methods have achieved good results in event prediction tasks. In these methods, graphs are constructed based on quadruples, and graph neural networks are used to capture pairwise correlations between entities. Although methods based on graph neural networks and recurrent neural networks have made significant progress, existing models are still limited in capturing high-order correlations between entities and their temporal evolution. High-order correlations refer to complex associations between three or more entities that exist simultaneously, which often contain deeper social logic and dynamic laws. To break through this bottleneck, the latest methods try to introduce derived structures such as entity groups, communities, and hypergraphs to characterize the relationships between entities in a more fine-grained manner. However, these methods cannot capture the temporal evolution of high-order correlations between entities. Summary of the invention

[0005] In view of the above, the purpose of the present invention is to provide a news event prediction method based on deep evolutionary clustering with time smoothing constraints, construct an entity graph according to the news events occurring at each timestamp, use the deep evolutionary clustering module to construct clusters, align and fuse the clusters of adjacent timestamps to construct a cluster graph, and then construct a time smoothing constraint, and introduce a cluster graph information transfer module for representation learning, so as to effectively model the temporal evolution of high-order correlations between entities, which has broad application prospects in the fields of international event prediction and social governance.

[0006] In order to achieve the above-mentioned invention object, the technical solution provided by the present invention is as follows:

[0007] The present invention provides a news event prediction method based on time smoothing constrained deep evolutionary clustering, comprising the following steps:

[0008] In the constructed training data set, an entity graph is constructed with entities as nodes and relationships between entities as edges according to news events occurring at each timestamp;

[0009] Use a relationship-aware graph convolutional neural network to aggregate information on the entity graph to obtain a preliminary updated entity representation and relationship representation;

[0010] In the deep evolutionary clustering module, based on the initially updated entity representation, fuzzy clustering is used to obtain the soft membership matrix of the entity to the cluster and the clusters of different timestamps. The clusters of adjacent timestamps are aligned and fused to construct a cluster graph with clusters as nodes and implicit correlations between clusters as edges. The temporal smoothing loss is constructed based on the cosine distance between clusters of adjacent timestamps.

[0011] In the cluster graph information transfer module, the implicit correlation encoder is used to capture the implicit correlation between cluster pairs in the cluster graph, and then information aggregation is performed on the cluster graph to update the cluster representation, and then the initially updated entity representation and relationship representation are updated again according to the soft membership matrix of the entity to the cluster;

[0012] The entity representation and relationship representation updated again at the current timestamp are combined with the initial entity representation and relationship representation at the current timestamp using the time residual gate as the input for the next timestamp;

[0013] The entity representation and relationship representation updated again with different timestamps are input into the time-dependent encoder based on the attention mechanism to obtain the comprehensive entity representation and relationship representation;

[0014] Based on comprehensive entity representation and relationship representation, all possible future relationships between entities are predicted through a convolution-based news event model. The total prediction loss including prediction loss and time smoothing loss is constructed. The entire network is trained based on the training dataset and the total prediction loss, and the trained news event model is used to predict news events.

[0015] Specifically, the entity graph with entities as nodes and relationships between entities as edges is constructed according to the news events occurring at each timestamp in the constructed training data set, including:

[0016] The news event data is preprocessed and the extracted news events are represented in the form of a four-tuple (s, r, o, t), where s represents the head entity, r represents the relationship, o represents the tail entity, and t represents the timestamp;

[0017] The preprocessed news event data is divided into training data sets using a sliding time window of size T, where each sample contains news events occurring at T timestamps in the news event data;

[0018] All news events in the training dataset are constructed into a global static graph, where nodes represent entities and edges represent relationships between entities. Spectral clustering is used to obtain global clusters based on the global static graph.

[0019] Divide the training dataset into N batches for batch training and randomly initialize the learnable entity representation e i and learnable relational representation r i, i represents the index of entity representation or relationship representation;

[0020] Select a batch of training samples and build an entity graph based on the news events occurring at each timestamp in, The set of nodes representing the timestamp t represents the entity; The edge set representing the timestamp t represents the relationship between entities.

[0021] Specifically, the use of a relationship-aware graph convolutional neural network to aggregate information on an entity graph to obtain a preliminary updated entity representation and relationship representation includes:

[0022] The calculation formula for information aggregation on the entity graph using the relationship-aware graph convolutional neural network is as follows:

[0023]

[0024] in, Indicates the preliminary updated tail entity representation. Since the tail entity in a news event can also serve as the head entity of another event, it is also considered that the head entity representation is synchronously and preliminarily updated here. and r t,l They represent the representation of the head entity s, tail entity o and relation r of timestamp t at the lth layer respectively. represents the set of news events at timestamp t, W1 and W2 represent learnable parameters, and d o represents the in-degree of the tail entity, and RReLU(·) represents the activation function;

[0025] t timestamp of the initial update of the relation representation r t Represented by the entity associated with r and the representation of r at the previous timestamp r t-1 The calculation formula is as follows:

[0026]

[0027] Among them, [;] represents the concatenation operation, pooling(·) represents the average pooling operation, Represents the entity representation of the initial update associated with r at timestamp t, where the entity set of the head entity or the tail entity x in (x,r,o,t) represents the head entity, and x in (s,r,x,t) represents the tail entity.

[0028] Specifically, the method of obtaining the soft membership matrix of the entity to the cluster and the clusters of different time stamps by using fuzzy clustering based on the initially updated entity representation in the deep evolution clustering module includes:

[0029] In the deep evolutionary clustering module, the soft membership matrix U of the entity to the cluster is first obtained by optimizing the following objective function J. The calculation formula is as follows:

[0030]

[0031] Among them, N e and N c denote the number of entities and clusters respectively, i denotes the index of the entity, j denotes the index of the cluster, and the element in the i-th row and j-th column of the soft membership matrix U An entity representation representing a preliminary update at timestamp t Cluster Centroid The membership degree of , the superscript m represents the fuzzy smoothing hyperparameter, <·> represents the dot product operation, ‖·‖ represents the modulus of the vector;

[0032] Then we get the overlapping clusters of timestamp t, and the calculation formula is as follows:

[0033]

[0034] in, represents the representation of the jth cluster at timestamp t, which is weighted by the entity representation and the entity's membership to the cluster. Represents the cluster centroid The cluster centroids with indices other than k, Represents a set of nodes in an entity graph.

[0035] Specifically, the step of aligning and fusing clusters of adjacent timestamps to construct a cluster graph with clusters as nodes and implicit correlations between clusters as edges includes:

[0036] In the deep evolutionary clustering module, the cluster-aware Hungarian matching algorithm is used to ensure smooth alignment of clusters of adjacent timestamps. It first constructs an affinity matrix A to quantify the similarity between clusters of consecutive timestamps. The calculation formula is as follows:

[0037]

[0038] Among them, the element a in the jth row and kth column of A j,k represents the jth cluster at timestamp t-1 and the kth cluster of timestamp t The similarity between them, cos(·) represents the cosine similarity function;

[0039] After the affinity matrix A is constructed, the cluster-aware Hungarian matching algorithm is used to seek the optimal alignment to maximize the sum of similarities between aligned clusters. The calculation formula is as follows:

[0040]

[0041] Where π represents a permutation function for achieving smooth alignment of clusters of adjacent timestamps, and π(j) represents the index of the cluster of timestamp t that is aligned with the jth cluster of timestamp t-1;

[0042] After aligning the clusters of adjacent timestamps, the aligned clusters are weighted fused to obtain the fused cluster representation. The calculation formula is as follows:

[0043]

[0044] Among them, β is the weight fusion hyperparameter, which indicates the relative contribution of cluster representations of adjacent timestamps to the fused cluster representation;

[0045] Then, construct the cluster graph in, The node set representing the timestamp t represents the cluster after alignment and fusion. The edge set at timestamp t represents the implicit correlation between the aligned and fused clusters.

[0046] Specifically, the calculation formula of the time smoothing loss is as follows:

[0047]

[0048] in, represents the temporal smoothing loss.

[0049] Specifically, the implicit correlation between cluster pairs in the cluster graph is captured by an implicit correlation encoder in the cluster graph information transmission module, and then information aggregation is performed on the cluster graph to update the representation of the cluster, and then the initially updated entity representation and relationship representation are updated again according to the soft membership matrix of the entity to the cluster, including:

[0050] The implicit correlation encoder in the cluster graph information transfer module is used to capture the implicit correlation between cluster pairs, the implicit correlation between the cluster pairs of the i-th cluster and the j-th cluster at timestamp t The calculation formula is as follows:

[0051]

[0052] in, represents the transformation function, which is implemented by a multi-layer perceptron, and ReLU(·) represents the activation function;

[0053] Quantifying the strength of implicit correlations The calculation formula is as follows:

[0054]

[0055] Among them, σ(·) represents the sigmoid function, Conv(·) represents the convolution operation;

[0056] Based on the global clusters obtained from the global static graph, the similarity between global clusters is used to enhance the strength of implicit correlation between cluster pairs. The calculation formula is as follows:

[0057]

[0058]

[0059] in, represents the global cluster aligned with the i-th cluster and the j-th cluster at timestamp t, represents the similarity between the ith global cluster and the ith global cluster at timestamp t, represents the implicit correlation strength of timestamp t after global cluster enhancement, which integrates the structural information from the global graph;

[0060] The calculation formula of the information transfer process of the cluster graph is as follows:

[0061]

[0062] in, Represents the time stamp t and the i-th cluster c i The corresponding vector representation;

[0063] Then, update the cluster representation and entity representation, and the calculation formula is as follows:

[0064]

[0065] in, represents the updated cluster representation of the ith cluster at timestamp t, represents the element in the i-th row and j-th column of the soft membership matrix, Indicates that t timestamp is N c The updated cluster representation of the jth cluster among the clusters is, It represents the updated entity representation of the i-th time stamp t, and obtains the updated relationship representation according to the updated entity representation.

[0066] Specifically, the convolution-based news event model adopts the ConvTransE model.

[0067] Specifically, the calculation formula for prediction loss is as follows:

[0068]

[0069] in, represents the prediction loss, N r represents the number of relationships, y i,j The label of the entity relationship j in event i, p i,jRepresents the predicted probability of entity j occurring in event i.

[0070] Specifically, the entire network is trained based on the training data set and the total prediction loss, including:

[0071] Divide the training data set into batches and calculate the prediction loss of each sample in the batch and time smoothing loss Added together as the total prediction loss for all samples in the batch Adjust the learnable parameters of the entire network, and the total prediction loss for all samples in the batch The calculation method is as follows:

[0072]

[0073] in, represents the prediction loss of the vth sample in the batch, F represents the number of samples in each batch, represents the temporal smoothing loss of the vth sample.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] (1) The present invention introduces a deep evolutionary clustering module, uses fuzzy clustering to obtain the soft membership matrix of entities to clusters, and uses a cluster-aware unsupervised alignment mechanism to align and fuse clusters of adjacent timestamps to construct a cluster graph. According to the cosine distance between clusters of adjacent timestamps, a temporal smoothing constraint is constructed to model the temporal evolution of high-order correlations between entities.

[0076] (2) The present invention introduces a cluster graph information transfer module, models the cluster graph as a fully connected graph, infers the implicit correlations between clusters in an end-to-end manner, and performs information aggregation and representation update based on the cluster graph, so as to utilize the temporal evolution of high-order correlations between entities for representation learning, thereby achieving accurate modeling of the temporal evolution of high-order correlations between entities. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0078] Figure 1 It is an overall flow chart of a news event prediction method based on time smoothing constrained deep evolutionary clustering provided by an embodiment of the present invention;

[0079] Figure 2It is an overall framework diagram of a news event prediction method based on time smoothing constrained deep evolutionary clustering provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0080] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0081] The inventive concept of the present invention is: in response to the problem that the temporal evolution of high-order correlations between entities cannot be captured in news event prediction tasks in the prior art, an embodiment of the present invention provides a news event prediction method based on deep evolutionary clustering with time smoothing constraints, constructs an entity graph according to news events occurring at each timestamp, uses a deep evolutionary clustering module to construct clusters, aligns and fuses clusters of adjacent timestamps to construct a cluster graph, and then constructs a temporal smoothing constraint, and introduces a cluster graph information transfer module for representation learning to model the temporal evolution of high-order correlations between entities.

[0082] Figure 1 is an overall flow chart of a news event prediction method based on time smoothing constrained deep evolutionary clustering provided by an embodiment of the present invention, Figure 2 This is an overall framework diagram of a news event prediction method based on time smoothing constraint deep evolution clustering provided by an embodiment of the present invention. Figure 1 and Figure 2 As shown, the embodiment provides a news event prediction method based on time smoothing constraint deep evolution clustering, comprising the following steps:

[0083] Step 1: Preprocess the news event data and extract news events. Use a sliding time window of size T to divide the processed data to obtain a training data set, where each sample contains news events that occurred at T timestamps in the news event data.

[0084] The news event data contains a large number of news events with time stamps, which are extracted. News events are represented in the form of a four-tuple (s, r, o, t), where s represents the head entity, r represents the relationship, o represents the tail entity, and t represents the timestamp. ε represents the entity set, mainly including entities such as country, region, citizen and company, and s,o∈ε. A collection of representative relationships, including cooperation, visits, statements, condemnations, and rejections, and represents a collection of timestamps, and

[0085] The timestamp of the quadruple is the time when the news event occurs. The quadruple is arranged in time sequence, and then the processed data is divided using a sliding time window of size T to obtain the training data set. Represents a set of news events that occurred at timestamp t, where each sample contains a set of news events that occurred at T timestamps in the news data

[0086] Step 2: Construct all news events in the training dataset into a global static graph G global , where nodes represent entities, edges represent relationships between entities, and spectral clustering is used to obtain global clusters

[0087] In this step, all news events in the training dataset are constructed into a global static graph, and spectral clustering is used to obtain global clusters. The calculation formula is as follows:

[0088]

[0089] in, represents the i-th global cluster, and SC(·) represents the spectral clustering algorithm.

[0090] Step 3: Randomly initialize the learnable entity representation e according to the in-degree of the node in the global static graph i , randomly initialize the learnable relation representation r i , i represents the index of entity representation or relationship representation.

[0091] Step 4: Divide the training data set into batches according to a fixed batch size, with a total batch size of N.

[0092] Step 5: sequentially select a batch of training samples with index k from the training data set, where k∈{1,…,N}, and repeat steps 6-15 for each training sample in the batch.

[0093] Step 6: Build an entity graph based on the news events that occurred at each timestamp in, The set of nodes representing the timestamp t represents the entity; The edge set representing the timestamp t represents the relationship between entities.

[0094] Step 7, for timestamps t=1, 2, ..., T-1, iteratively execute steps 8-12.

[0095] Step 8: Use the relationship-aware graph convolutional neural network to aggregate information on the entity graph.

[0096] In this step, the calculation formula for information aggregation on the entity graph using the relationship-aware graph convolutional neural network is as follows:

[0097]

[0098] in, Indicates the preliminary updated tail entity representation. Since the tail entity in a news event can also serve as the head entity of another event, it is also considered that the head entity representation is synchronously and preliminarily updated here. and r t,l They represent the representation of the head entity s, tail entity o and relation r of timestamp t at the lth layer respectively. represents the set of news events at timestamp t, W1 and W2 represent learnable parameters, and d o represents the in-degree of the tail entity, and RReLU(·) represents the activation function. In the following content, for the sake of brevity, the superscript l of the variable will be omitted without causing ambiguity. The relationship representation r of the initial update of the timestamp t Represented by the entity associated with r and the representation of r at the previous timestamp r t-1 The calculation formula is as follows:

[0099]

[0100] Among them, [;] represents the concatenation operation, pooling(·) represents the average pooling operation, Represents the entity representation of the initial update associated with r at timestamp t, where the entity set of the head entity or the tail entity x in (x,r,o,t) represents the head entity, and x in (s,r,x,t) represents the tail entity.

[0101] Step 9: Use the fuzzy clustering in the deep evolutionary clustering module to obtain the soft membership matrix U of the entity pair cluster and the cluster of the t timestamp

[0102] In this step, the soft membership matrix U of the entity to cluster is first obtained by optimizing the following objective function J. The calculation formula is as follows:

[0103]

[0104] Among them, N e and N c Represent the number of entities and clusters respectively, i represents the index of the entity, j represents the index of the cluster, and the element of the i-th row and j-th column of the soft membership matrix U An entity representation representing a preliminary update at timestamp t Cluster Centroid The superscript m represents the fuzzy smoothing hyperparameter, the value of m is set to be greater than 1, <·> represents the dot product operation, and ‖·‖ represents the modulus of the vector.

[0105] Then the overlapping clusters of timestamp t are obtained, and the calculation formula is as follows:

[0106]

[0107] in, represents the representation of cluster j at timestamp t, which is weighted by the entity representation and the entity's membership to the cluster. Represents the cluster centroid The indices other than are the cluster centroids of k.

[0108] Step 10: Use the cluster-aware unsupervised alignment mechanism to align and fuse clusters of adjacent timestamps to construct a cluster graph. in, The set of nodes with timestamp t represents the cluster; E c t Represents the edge set at timestamp t, representing the implicit correlation between clusters, and obtains the temporal smoothing loss based on the cosine distance between clusters of adjacent timestamps as a temporal smoothing constraint.

[0109] In this step, the cluster-aware Hungarian matching algorithm is used to ensure smooth alignment of clusters of adjacent timestamps. First, an affinity matrix A is constructed to quantify the similarity between clusters of consecutive timestamps. The calculation formula is as follows:

[0110]

[0111] Among them, the element a in the jth row and kth column of A j,k represents the jth cluster at timestamp t-1 and the kth cluster of timestamp t The similarity between them, cos(·) represents the cosine similarity function.

[0112] After the affinity matrix A is constructed, the cluster-aware Hungarian matching algorithm is used to seek the optimal alignment to maximize the sum of similarities between aligned clusters. The process of seeking the optimal alignment can be regarded as an optimization problem, and the calculation formula is as follows:

[0113]

[0114] Wherein, π represents a permutation function for achieving smooth alignment of clusters of adjacent timestamps, and π(j) represents the index of the cluster of timestamp t that is aligned with the jth cluster of timestamp t-1.

[0115] After aligning the clusters of adjacent timestamps, the aligned clusters are weighted fused to obtain the fused cluster representation. The calculation formula is as follows:

[0116]

[0117] Among them, β is the weight fusion hyperparameter, which indicates the relative contribution of cluster representations of adjacent timestamps to the fused cluster representation.

[0118] Then, construct the cluster graph in, The node set with timestamp t represents the cluster after alignment and fusion; represents the edge set of timestamp t, which represents the implicit correlation between clusters after alignment and fusion. In addition, the temporal smoothing loss is obtained based on the cosine distance between clusters of adjacent timestamps. As a time smoothing constraint, the calculation formula is as follows:

[0119]

[0120] Step 11: Use the implicit correlation encoder in the cluster graph information transfer module to capture the implicit correlation between cluster pairs, and use the similarity between global clusters to enhance the strength of the implicit correlation between cluster pairs. Use a graph convolutional neural network to aggregate information on the cluster graph to update the cluster representation, and update the entity representation according to the soft membership matrix U of the entity to the cluster. And update the relationship representation according to the updated entity representation

[0121] In this step, the implicit correlation encoder in the cluster graph information transfer module is used to capture the implicit correlation between cluster pairs. The implicit correlation between the cluster pairs of the i-th cluster and the j-th cluster at timestamp t The calculation formula is as follows:

[0122]

[0123] in, represents the transformation function, which is implemented by a multi-layer perceptron, and ReLU(·) represents the activation function.

[0124] In addition, the strength of the implicit correlation is quantified to obtain The calculation formula is as follows:

[0125]

[0126] Among them, σ(·) represents the sigmoid function, which ensures that the resulting implicit correlation strength value is between 0 and 1, and Conv(·) represents the convolution operation.

[0127] A higher interaction frequency in a longer time span indicates a stronger implicit correlation strength. To this end, the global clusters obtained based on the global static graph use the similarity between global clusters to enhance the strength of implicit correlation between cluster pairs. The calculation formula is as follows:

[0128]

[0129] in, Represents the global cluster aligned with the i-th cluster and the j-th cluster at timestamp t, the similarity matrix The element value in represents the similarity between the ith global cluster and the ith global cluster at timestamp t, represents the implicit correlation strength of timestamp t after global cluster enhancement, which integrates the structural information from the global graph.

[0130] The calculation formula of the information transfer process of the cluster graph is as follows:

[0131]

[0132] in, Represents the time stamp t and the i-th cluster c i The corresponding vector representation.

[0133] Then, update the cluster representation and entity representation, and the calculation formula is as follows:

[0134]

[0135] in, represents the updated cluster representation of the ith cluster at timestamp t, represents the element in the i-th row and j-th column of the soft membership matrix, Indicates that t timestamp is N c The updated cluster representation of the jth cluster among the clusters is, It represents the updated entity representation of the ith time stamp t, and obtains the updated relation representation according to the updated entity representation. In the following content, for the sake of brevity, the subscripts i and j of the variables will be omitted.

[0136] Step 12, using the temporal residual gate, combines the updated entity and relationship representations output at timestamp t with the entity and relationship representations of the input model at timestamp t to generate the input of the model at timestamp t+1.

[0137] In this step, the updated entity and relation representations output at timestamp t are combined with the representations of the input model at timestamp t using the temporal residual gate, which is calculated as follows:

[0138]

[0139] in, represents the dot multiplication operation, is the updated entity or relation representation output at timestamp t, H t-1 is the updated entity or relation representation output at timestamp t-1, i.e., the representation of the input model at timestamp t, X t is the time residual gate, which determines the proportion of information to be retained. The calculation formula is as follows:

[0140] X t =σ(W3H t-1 +b) (19)

[0141] Among them, σ is the sigmoid function, W3 and b are learnable parameters.

[0142] Step 13: Input the updated entity and relationship representations at different timestamps into the time-dependent encoder based on the attention mechanism to obtain the comprehensive entity and relationship representations.

[0143] In this step, a time-dependent encoder based on the attention mechanism is introduced to model the temporal dependency between representations. The calculation formula of the representation enhanced by position embedding is as follows:

[0144] z t =[H t ;Φ(t)] (20)

[0145]

[0146] Among them, [;] represents the concatenation operation, Φ(t) is the time position encoder, d is the dimension of representation, {ω i} i＝1,2,…,d is a learnable parameter, τ t is a timestamp. Then, given the representation sequence z 1:T-1 ={z 1 ,z 2 ,…,z T-1}, using the self-attention mechanism to capture the temporal dependencies between position-enhanced representations and output a comprehensive entity representation and comprehensive relationship representation

[0147] Step 14: Use the convolution-based news event model ConvTransE to predict all possible relationships between entities at timestamp T, and use the cross entropy function to calculate the prediction loss of a single sample.

[0148] Given a header entity and tail entity The news event prediction task aims to predict the relationship of the Tth timestamp based on the news events in the historical window [1:T-1]. In this step, the convolution-based news event model ConvTransE is used to predict the probability of different relationships between entities. The calculation formula is as follows:

[0149]

[0150] in, is the probability vector of the relationship, is the set of news events in the historical window [1:T-1], is a comprehensive entity representation, H r It is a comprehensive relationship representation matrix, each row corresponds to a comprehensive relationship representation ConvTransE(·) represents a one-dimensional convolutional layer and a fully connected layer.

[0151] After that, the cross entropy loss of a single sample is calculated based on the prediction results of all relations. The calculation formula is as follows:

[0152]

[0153] in represents the prediction loss, N r represents the number of relationships, y i,j The label of the entity relationship j in event i, y i,j ∈{0,1},p i,j Represents the predicted probability of entity relationship j occurring in event i, which is calculated by formula (22).

[0154] Step 15: The prediction loss of each sample in the batch and time smoothing loss Added together as the total prediction loss for all samples in the batch Adjust the learnable parameters of the entire network.

[0155] In this step, the prediction loss for all samples in the batch is calculated as follows:

[0156]

[0157] in, represents the prediction loss of the vth sample in the batch, F represents the number of samples in each batch, represents the time smoothing loss of the vth sample. Afterwards, according to the prediction loss Adjust the learnable parameters of the entire network.

[0158] Step 16: Repeat steps 5-15 until all batches of the training dataset participate in model training.

[0159] Step 17, repeat steps 5-16 until the specified number of iterations is reached.

[0160] Step 18: Input the processed data samples into the trained convolution-based news event model ConvTransE to obtain all possible future relationships between given entities.

[0161] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A news event prediction method based on time smoothing constrained deep evolutionary clustering, characterized in that: The following steps are involved: In the constructed training data set, an entity graph is constructed with entities as nodes and relationships between entities as edges according to news events occurring at each timestamp; Use a relationship-aware graph convolutional neural network to aggregate information on the entity graph to obtain a preliminary updated entity representation and relationship representation; In the deep evolutionary clustering module, based on the initially updated entity representation, fuzzy clustering is used to obtain the soft membership matrix of the entity to the cluster and the clusters of different timestamps. The clusters of adjacent timestamps are aligned and fused to construct a cluster graph with clusters as nodes and implicit correlations between clusters as edges. The temporal smoothing loss is constructed based on the cosine distance between clusters of adjacent timestamps. In the cluster graph information transfer module, the implicit correlation encoder is used to capture the implicit correlation between cluster pairs in the cluster graph, and then information aggregation is performed on the cluster graph to update the cluster representation, and then the initially updated entity representation and relationship representation are updated again according to the soft membership matrix of the entity to the cluster; The entity representation and relationship representation updated again at the current timestamp are combined with the initial entity representation and relationship representation at the current timestamp using the time residual gate as the input for the next timestamp; The entity representation and relationship representation updated again with different timestamps are input into the time-dependent encoder based on the attention mechanism to obtain the comprehensive entity representation and relationship representation; Based on comprehensive entity representation and relationship representation, all possible future relationships between entities are predicted through a convolution-based news event model. The total prediction loss including prediction loss and time smoothing loss is constructed. The entire network is trained based on the training dataset and the total prediction loss, and the trained news event model is used to predict news events.

2. The news event prediction method based on time smoothing constraint deep evolutionary clustering according to claim 1 is characterized in that: The entity graph with entities as nodes and relationships between entities as edges is constructed according to the news events occurring at each timestamp in the constructed training data set, including: The news event data is preprocessed and the extracted news events are represented in the form of a four-tuple (s, r, o, t), where s represents the head entity, r represents the relationship, o represents the tail entity, and t represents the timestamp; The preprocessed news event data is divided into training data sets using a sliding time window of size T, where each sample contains news events occurring at T timestamps in the news event data; All news events in the training dataset are constructed into a global static graph, where nodes represent entities and edges represent relationships between entities. Spectral clustering is used to obtain global clusters based on the global static graph. Divide the training dataset into N batches for batch training and randomly initialize the learnable entity representation e i and learnable relational representation r i , i represents the index of entity representation or relationship representation; Select a batch of training samples and build an entity graph based on the news events occurring at each timestamp in, The set of nodes representing the timestamp t represents the entity; The edge set representing the timestamp t represents the relationship between entities.

3. The news event prediction method based on time smoothing constraint deep evolutionary clustering according to claim 1 is characterized in that: The method uses a relationship-aware graph convolutional neural network to aggregate information on an entity graph to obtain a preliminary updated entity representation and relationship representation, including: The calculation formula for information aggregation on the entity graph using the relationship-aware graph convolutional neural network is as follows: in, Indicates the preliminary updated tail entity representation. Since the tail entity in a news event can also serve as the head entity of another event, it is also considered that the head entity representation is synchronously and preliminarily updated here. and r t,l They represent the representation of the head entity s, tail entity o and relation r of timestamp t at the lth layer respectively. represents the set of news events at timestamp t, W1 and W2 represent learnable parameters, and d o represents the in-degree of the tail entity, and RReLU(·) represents the activation function; t timestamp of the initial update of the relation representation r t Represented by the entity associated with r and the representation of r at the previous timestamp r t-1 The calculation formula is as follows: Among them, [;] represents the concatenation operation, pooling(·) represents the average pooling operation, Represents the entity representation of the initial update associated with r at timestamp t, where the entity set of the head entity or the tail entity x in (x,r,o,t) represents the head entity, and x in (s,r,x,t) represents the tail entity.

4. The news event prediction method based on time smoothing constraint deep evolutionary clustering according to claim 1 is characterized in that: The method of obtaining the soft membership matrix of the entity to the cluster and the clusters of different time stamps by using fuzzy clustering based on the initially updated entity representation in the deep evolution clustering module includes: In the deep evolutionary clustering module, the soft membership matrix U of the entity to the cluster is first obtained by optimizing the following objective function J. The calculation formula is as follows: Among them, N e and N c denote the number of entities and clusters respectively, i denotes the index of the entity, j denotes the index of the cluster, and the element in the i-th row and j-th column of the soft membership matrix U An entity representation representing a preliminary update at timestamp t Cluster Centroid The membership degree of , the superscript m represents the fuzzy smoothing hyperparameter, <·> represents the dot product operation, ‖·‖ represents the modulus of the vector; Then we get the overlapping clusters of timestamp t, and the calculation formula is as follows: in, represents the representation of the jth cluster at timestamp t, which is weighted by the entity representation and the entity's membership to the cluster. Represents the cluster centroid The cluster centroids with indices other than k, Represents a set of nodes in an entity graph.

5. The news event prediction method based on time smoothing constraint deep evolutionary clustering according to claim 1 is characterized in that: The step of aligning and fusing clusters of adjacent timestamps to construct a cluster graph with clusters as nodes and implicit correlations between clusters as edges includes: In the deep evolutionary clustering module, the cluster-aware Hungarian matching algorithm is used to ensure smooth alignment of clusters of adjacent timestamps. It first constructs an affinity matrix A to quantify the similarity between clusters of consecutive timestamps. The calculation formula is as follows: Among them, the element a in the jth row and kth column of A j,k represents the jth cluster at timestamp t-1 and the kth cluster of timestamp t The similarity between them, cos(·) represents the cosine similarity function; After the affinity matrix A is constructed, the cluster-aware Hungarian matching algorithm is used to seek the optimal alignment to maximize the sum of similarities between aligned clusters. The calculation formula is as follows: Where π represents a permutation function for achieving smooth alignment of clusters of adjacent timestamps, and π(j) represents the index of the cluster of timestamp t that is aligned with the jth cluster of timestamp t-1; After aligning the clusters of adjacent timestamps, the aligned clusters are weighted fused to obtain the fused cluster representation. The calculation formula is as follows: Among them, β is the weight fusion hyperparameter, which indicates the relative contribution of cluster representations of adjacent timestamps to the fused cluster representation; Then, construct the cluster graph in, The node set representing the timestamp t represents the cluster after alignment and fusion. The edge set at timestamp t represents the implicit correlation between the aligned and fused clusters.

6. The news event prediction method based on time smoothing constraint deep evolutionary clustering according to claim 5 is characterized in that: The calculation formula of time smoothing loss is as follows: in, represents the temporal smoothing loss.

7. The news event prediction method based on time smoothing constraint deep evolutionary clustering according to claim 2 is characterized in that: The method comprises: capturing the implicit correlation between cluster pairs in the cluster graph by using an implicit correlation encoder in the cluster graph information transmission module, performing information aggregation on the cluster graph to update the cluster representation, and then re-updating the initially updated entity representation and relationship representation according to the soft membership matrix of the entity to the cluster, including: The implicit correlation encoder in the cluster graph information transfer module is used to capture the implicit correlation between cluster pairs, the implicit correlation between the cluster pairs of the i-th cluster and the j-th cluster at timestamp t The calculation formula is as follows: in, represents the transformation function, which is implemented by a multi-layer perceptron, and ReLU(·) represents the activation function; Quantifying the strength of implicit correlations The calculation formula is as follows: Among them, σ(·) represents the sigmoid function, Conv(·) represents the convolution operation; Based on the global clusters obtained from the global static graph, the similarity between global clusters is used to enhance the strength of implicit correlation between cluster pairs. The calculation formula is as follows: in, and represents the global cluster aligned with the i-th cluster and the j-th cluster at timestamp t, represents the similarity between the ith global cluster and the ith global cluster at timestamp t, represents the implicit correlation strength of timestamp t after global cluster enhancement, which integrates the structural information from the global graph; The calculation formula of the information transfer process of the cluster graph is as follows: in, Represents the time stamp t and the i-th cluster c i The corresponding vector representation; Then, update the cluster representation and entity representation, and the calculation formula is as follows: in, represents the updated cluster representation of the ith cluster at timestamp t, represents the element in the i-th row and j-th column of the soft membership matrix, Indicates that t timestamp is N c The updated cluster representation of the jth cluster among the clusters is, It represents the updated entity representation of the i-th time stamp t, and obtains the updated relationship representation according to the updated entity representation.

8. The news event prediction method based on time smoothing constraint deep evolutionary clustering according to claim 1 is characterized in that: The convolution-based news event model adopts the ConvTransE model.

9. The news event prediction method based on time smoothing constraint deep evolutionary clustering according to claim 1 is characterized in that: The prediction loss is calculated as follows: in, represents the prediction loss, N r represents the number of relationships, y i,j The label of the entity relationship j in event i, p i,j Represents the predicted probability of entity j occurring in event i.

10. The news event prediction method based on time smoothing constraint deep evolutionary clustering according to claim 1 is characterized in that: The entire network is trained based on the training data set and the total prediction loss, including: Divide the training data set into batches and calculate the prediction loss of each sample in the batch and time smoothing loss Added together as the total prediction loss for all samples in the batch Adjust the learnable parameters of the entire network, and the total prediction loss for all samples in the batch The calculation method is as follows: in, represents the prediction loss of the vth sample in the batch, F represents the number of samples in each batch, represents the temporal smoothing loss of the vth sample.

Images