Event risk indicator association relationship prediction method based on multi-level graph representation learning

Abstract

The invention discloses an event risk indicator association relationship prediction method based on multi-level graph representation learning, and belongs to the technical field of graph neural network model design. The method comprises the following steps: collecting events within a certain time and corresponding risk assessment index weights and hit scores; the method comprises the following steps: training a Word2Vec model to convert text data into vectors, constructing a graph structure data sample, fusing semantic information of a text and graph embedding feature representation learning of multi-level indexes by using a graph attention network, carrying out risk assessment and prediction by using feature representation of the lowest-level index, and carrying out multi-level index weight updating based on risk true value loss. Meanwhile, a mask strategy is introduced to remove the influence of an irrelevant relation, and a model is trained; and performing event risk index association relationship prediction by using the trained model. According to the method, the incidence relation between the waiting indexes can be fully mined, and the precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of graph neural network model design, and in particular relates to a method for predicting the correlation relationship of event risk indicators based on multi-level graph representation learning. Background Art

[0002] In the field of risk assessment, risk is defined as the probability of harmful consequences or expected losses resulting from the complex interactions between hazards and vulnerabilities in cities and societies. In recent years, risk assessment, particularly multi-indicator risk analysis, has become a hot topic in risk analysis due to its comprehensive and integrated consideration of various factors. Existing methods primarily include those based on fuzzy graphs and Bayesian networks. These methods primarily rely on qualitative correlations between indicators, using fuzzy relationships or probabilistic statistical analysis to introduce influencing factors between adjacent nodes in the graph to achieve measurement.

[0003] Not everything in the real world can be represented as a sequence or a grid. For example, social networks, knowledge graphs, complex file systems, and so on, are all unstructured. The difficulties in processing them include: the size of the graph is arbitrary, the topology of the graph is complex, and there is no spatial locality like images; the graph does not have a fixed node order, or a reference node; the graph is often dynamic and contains multimodal features. These problems have led to the emergence and development of graph neural networks (GNNs). A common application model of GNNs is as follows: the input is a graph, and after various operations such as multi-layer graph convolution and activation functions, the representation of each node is finally obtained to facilitate tasks such as node classification, link prediction, and graph and subgraph generation.

[0004] There are some problems with existing methods based on fuzzy graphs and Bayesian networks. The first is the problem of indicator information redundancy. Current multi-indicator association analysis methods generally face the challenge of indicator information redundancy when dealing with the evolution process of thematic events. Although traditional dimensionality reduction methods can extract principal components, their linear assumptions make it difficult to deal with nonlinear redundant relationships in the evolution process of events. Although the autoencoder in deep learning can achieve nonlinear dimensionality reduction, its hidden layer features often lack clear semantic interpretability. Second, the dilemma of indicator coupling analysis. The complex coupling relationship between multiple indicators causes the traditional association analysis method to fail. Such as time-varying coupling, hierarchical coupling and implicit coupling. Existing solutions such as dynamic Bayesian networks (DBN) can model temporal dependencies, but the learning complexity of their network structure is O(T·n 2) (T is time, n is the number of indicators), making it difficult to handle high-dimensional indicator systems. Finally, there is the issue of semantic ambiguity. The ambiguity of indicator representation of events is mainly manifested in: semantic drift, where the same indicator conveys different meanings at different stages of an event (such as the different interpretations of "trading volume" in the stock market during bull and bear markets); time-varying contribution: the contribution of key indicators follows a piecewise function over time; and cross-domain heterogeneity: the lack of a unified quantitative framework for physical and semantic indicators (such as the correlation between sensor data and public opinion text). Summary of the Invention

[0005] To address the above issues, the present invention proposes a method for predicting the correlation between event risk indicators based on multi-level graph representation learning. Taking into account the importance of the semantic information of indicators and multi-level indicators for event risk assessment, the present invention utilizes a graph attention network to learn the feature representation of multi-level indicators. The final prediction score is learned by integrating the lowest-level indicators. During the updating and learning process, a masking strategy is used to limit the learning of redundant correlations in the graph neural network, such as the relationship between the next-level indicators of the same-level indicators and the current-level indicators. This method can effectively learn the correlation between event risk indicators.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] The method for predicting the correlation relationship of event risk indicators based on multi-level graph representation learning includes the following steps:

[0008] Step 1: Build a training set;

[0009] Step 2: Construct a multi-level graph representation learning event risk indicator correlation relationship prediction network;

[0010] Step 3: Use the samples in the training set to train the event risk indicator correlation relationship prediction network learned by multi-level graph representation;

[0011] Step 4: Use the trained multi-level graph representation to learn the event risk indicator correlation relationship prediction network to predict the event risk indicator correlation relationship and obtain the correlation weight.

[0012] Furthermore, the specific method of step 1 is:

[0013] Collect original records of multiple historical events that occurred within a period of months;

[0014] Extracting core information fields from each collected raw event record; the core information fields include at least: a natural language event description text recording the specific content of the event, a quantitative weight value corresponding to one or more predefined risk assessment indicators, and a quantitative score value indicating the degree to which the event triggers a specific risk rule or matches a risk feature;

[0015] Use the pre-trained Word2Vec model to learn semantic knowledge from the text data in the dataset in an unsupervised manner and convert the event description text into vectors;

[0016] Graph structure data is constructed based on the vector transformed from the event description text. One graph structure data is obtained for each month. The graph structure data is defined as G = (V, E), where node v i ∈V represents the events of the month and the multi-level evaluation indicators, and the edge e i,j ∈E represents the relationship between nodes, and the adjacency matrix A is used to record the topological structure of G. The elements in A satisfy the following formula:

[0017]

[0018] in, <v i ,v j >∈E represents v i ,v j The line connecting is an edge in E;

[0019] All graph structure data constitute the training set.

[0020] Furthermore, the event risk indicator association relationship prediction network of the multi-level graph representation learning described in step 2 includes a multi-level graph representation learning module and a lowest-level fusion prediction module; wherein the multi-level graph representation learning module includes two multi-head graph attention convolution layers, the hidden space dimension size is 128, the input channel of the first multi-head attention convolution layer is the graph data dimension, and the output channel is the hidden space dimension, and the input and output channels of the second convolution layer are both hidden space dimensions; the multi-level graph representation learning module processes the monthly graph structure data, performs graph embedding feature representation learning of multi-level indicators, and obtains a multi-level indicator graph; the lowest-level fusion prediction module includes a projection layer, which fuses the graph embedding feature representation of the lowest-level indicator node of the multi-level indicator graph to obtain the overall risk score;

[0021] The processing method of the event risk indicator correlation relationship prediction network learned by multi-level graph representation is as follows:

[0022] The graph structure data is input into the multi-level graph representation learning module. The multi-head attention convolution layer in the multi-level graph representation learning module implements the self-attention mechanism for each node. The attention coefficient is:

[0023]

[0024] Among them, a represents a single-layer feedforward neural network, W represents the weight matrix, and the vector Represents the feature vector of the node, subscripts i and j represent the i-th node and the j-th node;

[0025] Through the softmax function i,j Perform regularization:

[0026]

[0027] Among them, N i represents the neighbor node set of node i, α ij Represents the attention coefficient between nodes after regularization;

[0028] Finally, the feature representation of node i is:

[0029]

[0030] Among them, σ is a nonlinear activation function;

[0031] During the parameter update process, some positions in the adjacency matrix A are masked, that is, the positions corresponding to node i and node j are set to 0: A[i][j] = 0 and A[j][i] = 0;

[0032] After two graph attention convolution layers, a multi-level graph embedding representation is obtained

[0033] The lowest level fusion prediction module accepts the multi-level graph embedding representation learned by the multi-level graph representation learning module The graph embedding feature representations of the lowest level indicator nodes of the multi-level indicator graph are fused to obtain the overall risk score:

[0034]

[0035] in, represents the predicted risk score, Z n-i: Represents the feature representation of all lowest-level index nodes i.

[0036] Furthermore, in step 3, MSE is used as the loss function, the loss function gradient is calculated through back propagation, and the parameters are updated using an adaptive optimization algorithm; at the end of each training cycle, the model performance is evaluated based on an independent validation set, and the training process continues until the maximum number of iterations is reached, and then the learning process is terminated. The model parameters with the best performance in the entire training process are automatically identified and stored through a global comparison mechanism.

[0037] Furthermore, the specific method of step 4 is:

[0038] Collect multiple events within at least one month, along with their descriptions, weights, and hit scores under evaluation indicators;

[0039] The text content in the data is converted into word embeddings using a pre-trained Word2Vec model, and the corresponding monthly graph structure data is constructed, where nodes represent events and multi-level evaluation indicators, and edges represent the relationships between nodes;

[0040] The processed graph structure data is input into the trained multi-level graph representation learning event risk indicator correlation prediction network to obtain the risk score of the current month, and an association matrix is ​​output at the same time. The element value of the association matrix is ​​the association weight between the standardized lowest level indicators.

[0041] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0042] 1. The present invention takes into account that the semantic information of indicators and multi-level indicators are crucial to the assessment of event risks. It uses a graph attention network to learn the feature representation of multi-level indicators, and learns the final prediction score by fusing the lowest-level indicators. During the updating and learning process, a masking strategy is used to limit the learning of redundant associations in the graph neural network, such as the relationship between the next-level indicators of the same-level indicators and the current-level indicators. This can effectively learn the association between event risk indicators.

[0043] 2. The present invention uses the word2Vec model to convert the text information of the event into a vector, which can better represent the event. At the same time, it uses a graph attention network to learn the graph embedding feature representation of multi-level indicators, where the calculation of node-neighbor pairs can be parallelized, so the operation efficiency is very high.

[0044] 3. During the model update process, the present invention uses a masking strategy to limit the learning of redundant association relationships, such as the relationship between the next-level indicators of the same-level indicators and the current-level indicators, so that the final evaluation results are more accurate and more suitable for the application scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the principle of the event risk indicator correlation prediction method based on multi-level graph representation learning.

[0046] Figure 2 This is a schematic diagram of the Word2Vec structure.

[0047] Figure 3 It is a schematic diagram of the graph data structure.

[0048] Figure 4 This is a schematic diagram of the graph attention network.

[0049] Figure 5 It is a schematic diagram of masking some positions in the adjacency matrix. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical methods and advantages of the present invention more clearly understood, the present invention is further described below with reference to the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention in any way.

[0051] An event risk indicator correlation relationship prediction method based on multi-level graph representation learning is proposed. The method first collects events within a certain period of time and their corresponding risk assessment indicator weights and hit scores; then trains the Word2Vec model to convert text data into vectors, constructs graph structure data samples, and uses a graph attention network to fuse the semantic information of the text and the graph embedding feature representation learning of multi-level indicators. The feature representation of the lowest-level indicator is used for risk assessment and prediction, and the multi-level indicator weights are updated based on the true value loss of the risk. At the same time, a masking strategy is introduced to remove the influence of irrelevant relationships and train the model; finally, the trained model is used to predict the event risk indicator correlation relationship.

[0052] like Figure 1 As shown in the figure, taking a risk event as an example, the specific steps of this method are:

[0053] Step 1, data preparation: Collect multiple events that affect a certain event within a certain period of time and their corresponding risk assessment indicator weights and hit scores.

[0054] The current events and evaluation indicators are all text data, and the detection scope of risk level assessment is clarified. It is necessary to learn semantic knowledge from text data in an unsupervised manner, and convert events and indicators and other uncomputable and unstructured texts into computable and structured vectors. Word2Vec is a good tool for generating word vectors. Word vectors are a way to mathematize words in a language. In the input layer, Word2Vec is a lightweight neural network. Its model only includes input layer, hidden layer and output layer. The simplest idea is to convert a word into a One-Hot vector. Then, in the first hidden layer, the input is Wx+b (x is the input word vector, W, b are parameters). Note that this is just a simple mapping and there is no nonlinear activation function. The third layer can be simply regarded as a classifier, using Softmax regression, and the final output is the probability corresponding to each word, such as Figure 2 It is a simple structure of Word2Vec.

[0055] This embodiment uses the Word2Vec model to convert events and evaluation indicators into 128-dimensional vectors to facilitate the subsequent construction of the graph structure, and simultaneously splices weights and hit scores, filling them with 0 if they are missing.

[0056] Using data samples, we construct graph structure data samples with months as the time unit. A graph is defined as G = (V, E), with node v i ∈V represents events and multi-level evaluation indicators, and edge ei,j ∈E represents the relationship between nodes, and the adjacency matrix A is used to record the topological structure of G. The elements in A satisfy the following formula:

[0057]

[0058] When A[i][j] is 0, it represents vertex V i and V j There is an association relationship between them, otherwise it is 0;

[0059] The graph data of an event may have the following structure: Figure 3 shown.

[0060] The main nodes are the events of the month, the first-level evaluation indicators, and the second-level evaluation indicators.

[0061] Step 2: Construct a multi-level graph representation learning event risk indicator correlation prediction network.

[0062] First, a graph attention network is used to introduce a self-attention mechanism in the propagation process to learn graph embedding feature representations of multi-level indicators.

[0063] like Figure 4 As shown in the figure, the graph attention network introduces the attention mechanism into the graph neural network based on the spatial domain. Unlike the graph convolutional neural network based on the spectral domain introduced previously, the graph attention network does not need to use matrices such as Laplace for complex calculations, but only updates the node features through the representation of neighboring nodes.

[0064] The graph attention network consists of a graph attention layer, which implements a self-attention mechanism for each node. The attention coefficient is:

[0065]

[0066] Among them, the vector h is the characteristic vector of the node, and the subscripts i and j represent the i-th node and the j-th node.

[0067] In order to make the attention coefficient easier to calculate and compare, softmax is introduced to regularize all the adjacent nodes j of i:

[0068]

[0069] Among them, N i Refers to the set of neighbor nodes of node i, where attention is allocated to the neighbor nodes of each node through masked attention;

[0070] Where a is a single-layer feedforward neural network. A LeakyReLu function is added to the output layer of the feedforward neural network to give a non-zero slope to all negative values.

[0071] In summary, the complete attention mechanism is as follows:

[0072]

[0073] The above operations give the regularized attention coefficients between different nodes, which can be used to learn the feature representation of each graph node:

[0074]

[0075] Among them, W is the weight matrix multiplied by the feature, α ij is the attention correlation coefficient calculated previously, σ is a nonlinear activation function, and this formula indicates that the feature representation of each node is calculated by paying attention to its neighboring nodes and assigning corresponding weights to its neighbors. During the update process of each node, the features of all neighboring nodes are weightedly summed based on attention.

[0076] In addition, this layer also uses multi-head attention to stabilize the learning process. It applies K independent attention mechanisms to calculate the hidden state, and then concatenates (or averages) their features to obtain the following two output representations:

[0077]

[0078] in, is the normalized attention coefficient of the kth attention head, || represents the splicing operation, and the graph embedding feature representation using the graph attention network to learn multi-level indicators can handle nodes of different degrees and assign corresponding weights to their neighbors.

[0079] Here, the computation of node-neighbor pairs is parallelizable and thus highly efficient.

[0080] Here, the event risk level assessment is mainly based on the weight and hit score of the lowest-level indicator node. Then, the graph embedding feature representation of the lowest-level indicator node of the multi-level indicator graph is fused to obtain the overall risk score, as follows:

[0081]

[0082] in, represents the predicted risk score, W is the weight matrix, Z n-i: Represents the feature representation of all lowest-level index nodes i.

[0083] Step 3: Use the samples in the training set to train the event risk indicator correlation prediction network learned by multi-level graph representation, and update the multi-level indicator weights based on the risk true value loss: Use the weight matrix to fuse the graph embedding feature representation of the multi-level indicators into an overall risk score. The loss function is:

[0084]

[0085] The mean squared error (MSE) function measures the distance between a sample point and the regression curve, minimizing the squared loss to allow the sample point to better fit the regression curve. A smaller MSE value indicates a more accurate description of the sample data by the prediction model.

[0086] During the parameter update process, some positions in the adjacency matrix A are masked, that is, the positions corresponding to node i and node j are set to 0: A[i][j] = 0 and A[j][i] = 0. The idea is to limit the network's learning of the association relationship between the two nodes at the corresponding positions. Here, the next level indicator of the previous level indicator is irrelevant to the same level indicator, and the relationship between different events is ignored, such as Figure 5 shown.

[0087] Step 4, verification and evaluation: Use the trained model to test the data in the test set to obtain the correlation and weight between symptom indicators and the risk score of the event.

[0088] This method first collects data, converts text data into vectors by training the Word2Vec model, and constructs graph structure data samples; then uses the graph attention network to learn the graph embedding feature representation of multi-level indicators, uses the feature representation of the lowest-level indicators for risk assessment and prediction, and updates the weights of multi-level indicators through the mask strategy and risk true value loss. At the same time, the mask strategy is introduced to remove the influence of irrelevant relationships; finally, the trained model is used to obtain the final risk score prediction result.

[0089] In summary, this invention utilizes graph neural networks to establish a hierarchical indicator system, modeling the correlations between indicators. Pretrained language models enhance semantic information representation, and the attention mechanism within the graph network learns salient features, alleviating the problem of indicator information redundancy. Furthermore, graph neural networks enable objective assessment of temporal risk based on the importance of indicators, qualitatively analyzing and evaluating the correlations between indicators, and quantitatively calculating indicator weights. This invention fully exploits the correlations between symptom indicators, improving accuracy.

Claims

1. A method for predicting the correlation between event risk indicators based on multi-level graph representation learning, characterized by: The following steps are involved: Step 1: Build a training set; Step 2: Construct a multi-level graph representation learning event risk indicator correlation relationship prediction network; Step 3: Use the samples in the training set to train the event risk indicator correlation relationship prediction network learned by multi-level graph representation; Step 4: Use the trained multi-level graph representation to learn the event risk indicator correlation relationship prediction network to predict the event risk indicator correlation relationship and obtain the correlation weight.

2. The method for predicting event risk indicator correlation relationships based on multi-level graph representation learning according to claim 1 is characterized in that: The specific method of step 1 is: Collect original records of multiple historical events that occurred within a period of months; Extract core information fields from each collected original event record; The core information field includes at least: a natural language event description text recording the specific content of the event, a quantitative weight value corresponding to one or more predefined risk assessment indicators, and a quantitative score value indicating the degree to which the event triggers a specific risk rule or matches a risk feature; Use the pre-trained Word2Vec model to learn semantic knowledge from the text data in the dataset in an unsupervised manner and convert the event description text into vectors; Graph structure data is constructed based on the vector transformed from the event description text. One graph structure data is obtained for each month. The graph structure data is defined as G = (V, E), where node v i ∈V represents the events of the month and the multi-level evaluation indicators, and the edge e i,j ∈E represents the relationship between nodes, and the adjacency matrix A is used to record the topological structure of G. The elements in A satisfy the following formula: in, <v i ,v j >∈E represents v i ,v j The line connecting is an edge in E; All graph structure data constitute the training set.

3. The method for predicting event risk indicator correlation relationships based on multi-level graph representation learning according to claim 2 is characterized in that: The event risk indicator association relationship prediction network of the multi-level graph representation learning described in step 2 includes a multi-level graph representation learning module and a lowest-level fusion prediction module; wherein, the multi-level graph representation learning module includes two multi-head graph attention convolution layers, the hidden space dimension size is 128, the input channel of the first multi-head attention convolution layer is the graph data dimension, and the output channel is the hidden space dimension, and the input and output channels of the second convolution layer are both hidden space dimensions; the multi-level graph representation learning module processes the monthly graph structure data, performs graph embedding feature representation learning of multi-level indicators, and obtains a multi-level indicator graph; the lowest-level fusion prediction module includes a projection layer, which fuses the graph embedding feature representation of the lowest-level indicator node of the multi-level indicator graph to obtain the overall risk score; The processing method of the event risk indicator correlation relationship prediction network learned by multi-level graph representation is as follows: The graph structure data is input into the multi-level graph representation learning module. The multi-head attention convolution layer in the multi-level graph representation learning module implements the self-attention mechanism for each node. The attention coefficient is: Among them, a represents a single-layer feedforward neural network, W represents the weight matrix, and the vector Represents the feature vector of the node, subscripts i and j represent the i-th node and the j-th node; Through the softmax function i,j Perform regularization: Among them, N i represents the neighbor node set of node i, α ij Represents the attention coefficient between nodes after regularization; Finally, the feature representation of node i is: Among them, σ is a nonlinear activation function; During the parameter update process, some positions in the adjacency matrix A are masked, that is, the positions corresponding to node i and node j are set to 0: A[i][j] = 0 and A[j][i] = 0; After two graph attention convolution layers, a multi-level graph embedding representation is obtained The lowest level fusion prediction module accepts the multi-level graph embedding representation learned by the multi-level graph representation learning module The graph embedding feature representations of the lowest level indicator nodes of the multi-level indicator graph are fused to obtain the overall risk score: in, represents the predicted risk score, Z n-i : Represents the feature representation of all lowest-level index nodes i.

4. The method for predicting event risk indicator correlation relationships based on multi-level graph representation learning according to claim 3 is characterized in that: In step 3, MSE is used as the loss function, the loss function gradient is calculated through backpropagation, and the parameters are updated using an adaptive optimization algorithm. At the end of each training cycle, the model performance is evaluated based on an independent validation set. The training process continues until the maximum number of iterations is reached, and then the learning process is terminated. The model parameters that perform best in the entire training process are automatically identified and stored through a global comparison mechanism.

5. The method for predicting event risk indicator correlation relationships based on multi-level graph representation learning according to claim 4 is characterized in that: The specific method of step 4 is: Collect multiple events within at least one month, along with their descriptions, weights, and hit scores under evaluation indicators; The text content in the data is converted into word embeddings using a pre-trained Word2Vec model, and the corresponding monthly graph structure data is constructed, where nodes represent events and multi-level evaluation indicators, and edges represent the relationships between nodes; The processed graph structure data is input into the trained multi-level graph representation learning event risk indicator correlation prediction network to obtain the risk score of the current month, and an association matrix is ​​output at the same time. The element value of the association matrix is ​​the association weight between the standardized lowest level indicators.

Images