Security risk relation extraction method based on multi-view IB

Abstract

The invention discloses a security risk relation extraction method based on multi-view IB, which uses a deep neural network to automatically learn semantic features of security risks, and introduces a multi-view method to enhance domain entity characterization in order to solve the problem of insufficient entity characterization in traditional domain relation extraction. When the semantic information of the two view angles is fused, the maximum related information between the two view angles is obtained by adopting an information bottleneck method, and the fused semantics is expressed by using the maximum related information. According to the method, domain context semantics are understood through the heterogeneous graph neural network, and relationships between entities can be divided into different categories, so that the semantics of an image view angle can be fully utilized to enhance representation of domain entities, and a better domain relationship extraction result is ensured.

Description

Technical Field

[0001] The present invention belongs to the field of security risk management, and in particular relates to a security risk relationship extraction method based on multi-perspective IB. Background Art

[0002] In the academic and industrial fields, the wide application of knowledge graphs has attracted widespread attention, such as English knowledge graph, Microsoft knowledge graph, Yago, etc. In the field of information retrieval, these knowledge graphs have achieved good results. However, there are relatively few studies on knowledge graphs in specific fields.

[0003] The most important task in building a knowledge graph is relation extraction, which aims to extract all possible relation triplets (subject, predicate, object) in natural language sentences. As an upstream task in building a knowledge graph, this task plays an important role. With the continuous development of deep learning, people are increasingly interested in relation extraction based on deep learning, trying to use neural networks to automatically learn semantic features. Recently, a large number of domain relation extraction models have been proposed and applied to multiple fields such as biomedical relation extraction and KPI relation extraction.

[0004] Relation extraction is a basic task in the field of natural language processing (NLP), which can be divided into four categories: supervised relation extraction, semi-supervised relation extraction, unsupervised relation extraction, and remote supervised relation extraction. In recent years, most researchers have focused on supervised relation extraction and remote supervised relation extraction based on neural networks, both of which have achieved good results. With the widespread application of deep learning, relation extraction based on deep learning has become a hot topic of research.

[0005] Socher et al. first introduced RNN into the task of relation extraction, solving the serious problem that traditional methods cannot capture long dependencies with high quality. Sun et al. proposed to use the PCNN model for relation extraction and formed an entity relationship table in the power communication planning text. Zeng et al. proposed to use CNN for relation extraction and used convolutional neural network to extract local features of input sentences. Yan et al. combined the dependency analysis tree features with word vector features and part-of-speech features, achieving good results, but consuming more computing resources. However, since the aforementioned models are built on a sufficient amount of training data, their performance in relation extraction from sparse fields is always unsatisfactory. Wu et al. used graph GCN to extract deeper information to improve the effect of relation extraction. Zhuang et al. used knowledge enhancement to extract event relations and made good progress.

[0006] In 1999, Tishby et al. proposed the information bottleneck theory based on mutual information in order to achieve information compression and noise removal. The information bottleneck (IB) method studies the trade-off between compression and prediction: after extracting relevant information from the input variable X while retaining the relevant information of another random variable Y, the representation obtained is another variable Z. Recently, Tishby et al. analyzed the relationship between the information bottleneck method and deep neural networks, which has attracted widespread attention in the computer vision and natural language processing communities. In the NLP community, the information bottleneck method is mainly used for representation learning. Li et al. used variational information bottlenecks to nonlinearly compress BERT pre-trained word embeddings. Amjad used the information bottleneck method to study the representation of neural network-based classification. However, there is a lack of information bottleneck theory used in relation extraction.

[0007] Among all these domain relation extraction models, models based on deep neural networks have shown good performance in exploring and developing domain semantics. Although the above domain relation extraction models based on deep neural networks have achieved considerable results, they still face the following problems.

[0008] (1) Insufficient entity semantic representation: In the relation extraction models mentioned above, entity semantics are mostly obtained through pre-training models, which can be considered as a primary version. However, in domain relation extraction, it is often the case that there are no domain entities in the pre-training corpus, so the characteristics of the domain entities cannot be fully represented, which further affects the performance of identifying relation types.

[0009] (2) How to obtain the most relevant information between two perspectives: In the existing technology, most of the splicing methods are used to achieve semantic fusion. Although this approach improves performance, it also incorporates information that is not related to the entity. Summary of the invention

[0010] In order to address the shortcomings of the prior art, the present invention aims to provide a security risk relationship extraction method based on multi-perspective IB, which uses multi-perspective semantics to enhance entity semantics, and obtains the maximum relevant information between two perspectives through the information bottleneck theory. It can automatically extract semantic information from multiple corresponding images, thereby making full use of the semantics of image perspectives to enhance the representation of domain entities, thereby ensuring better domain relationship extraction results.

[0011] In order to achieve the above object, the technical solution adopted by the present invention is:

[0012] A security risk relationship extraction method based on multi-perspective IB includes the following steps:

[0013] 1) Use multi-view semantic fusion strategy to fuse text nodes and image information;

[0014] 2) Use the deep neural network RetNet to learn the information between text and pictures, so as to use the information between pictures to supplement the deficiencies in the semantic representation of text;

[0015] 3) Leveraging heterogeneous graph spaces to model word nodes and relation nodes to capture rich contextual semantics with more task-related features;

[0016] 4) Use information bottlenecks to compress information;

[0017] 5) Use the max-pooling mechanism to obtain the representation of the corresponding entity, use the concatenation operation to concatenate the entity information and sentence information together, and use the feedforward neural network FFN to obtain the final representation of the relationship classification; and input the obtained high-level semantics into the softmax classifier to calculate the conditional probability of the security risk relationship to which the entity belongs.

[0018] Wherein, step 1) specifically includes:

[0019] 1-1) Problem definition: Given multi-view data, let T = {x1, x2, …, x |T|} as the input sentence, where x i is a word, |T| represents the length of the sentence; the purpose of domain relation extraction is to identify entity pairs (x i1-i2 ,x j1-j2 ), x i1-i2 、x j1-j2 Respectively represent the span of two entities i1-i2 and j1-j2; the relationship set is

[0020] 1-2) Use multi-view semantic fusion strategy to encode and extract security risk knowledge;

[0021] First, the security risk text after word segmentation is used to obtain low-dimensional word vectors using the pre-trained language model BERT, specifically:

[0022] For an input sentence T, the word It is expressed as:

[0023]

[0024] in, d w is the dimension of the word vector; d c is the dimension of the position vector;

[0025] 1-3) In order to enhance the semantic information of domain entities, the image perspective semantic information is fused to supplement the insufficient representation in the pre-trained model. Specifically:

[0026] For domain entity x i , the image set corresponding to the match is m = {m i1 ,m i2 ,...,m in}, and then use VggNet to obtain the image feature vector m i , directly add the image feature vector to the corresponding entity feature.

[0027] Step 3) specifically includes:

[0028] The word x i Rewrite as word node The relationship i Rewrite as a relationship node And introduce a message passing mechanism to update the node representation in the following way:

[0029]

[0030] Among them, [·;·] means connecting two vectors into a new representation, represents the neighboring points of the i-th node, W a ,W q ,W k ,W v is the trainable weight; o j The encoding of the relationship, α ij represents the probability value of the attention mechanism, h i ′ represents the updated code, h i Indicates the encoding before updating.

[0031] Then, a gate mechanism is used instead of an activation function to maintain the scale and nonlinearity of each dimension, as shown below:

[0032]

[0033] Among them, W g is the trainable weight, v i is the probability of the gate mechanism, For the final code after passing through the door;

[0034] The above operations of heterogeneous graphs are summarized as follows:

[0035]

[0036] Similarly, the encoding of the relationship node is:

[0037]

[0038] In step 4), the idea of ​​first compressing and then restoring the information bottleneck is used to express the relevant information of the two perspectives as follows:

[0039]

[0040] Where ∈ is the standard normal distribution, N is the number of training examples, KL is the Kullback-Leibler divergence, L1 is the loss, I is the mutual information, β is the trade-off parameter, Y is the target encoding, Z is the intermediate encoding, H is the initial encoding, y, z and h are the specific variable observations, Y, Z and H represent the set of variables, p(.) represents the corresponding probability, and q(z) is a variational approximation with the same distribution as p(z).

[0041] In step 5), the conditional probability p(r|T,θ) of the security risk relationship to which the entity belongs is calculated, specifically:

[0042]

[0043] Where |R| is the number of security risk relationship types, O r It is the final output of the heterogeneous graph neural network, i.e., the prediction results of all security risk relationship types;

[0044] Use cross entropy for classification as follows:

[0045]

[0046] Wherein, L2 represents the loss, n is the number of input sentence sets, and θ is a set of parameters that the model of the present invention needs to learn;

[0047] The total loss of the model is:

[0048] L=L1+L2 (18)

[0049] In model training, the stochastic gradient descent algorithm SGD is used to optimize the objective function.

[0050] The present invention has the following beneficial effects:

[0051] The present invention uses a deep neural network to automatically learn the semantic features of security risks. In order to solve the problem of insufficient entity representation in traditional domain relationship extraction, the present invention introduces a multi-perspective method to enhance the representation of domain entities. When fusing the semantic information of two perspectives, the present invention uses the information bottleneck method to obtain the maximum relevant information between the two perspectives, which is used to represent the fused semantics. By understanding the domain context semantics through heterogeneous graph neural networks, the relationships between entities can be divided into different categories, so that the semantics of the image perspective can be fully utilized to enhance the representation of domain entities, thereby ensuring better domain relationship extraction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a multi-perspective IB security risk relationship extraction model diagram of the present invention;

[0053] Figure 2 and Figure 3 This is the precision-recall curve of different models of the present invention on the RVSR dataset. DETAILED DESCRIPTION

[0054] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not intended to limit the scope of use of the present invention.

[0055] Relation extraction is the key content of information extraction and a key step in building a security risk knowledge graph. In the context of big data, in order to solve the problems of inefficiency and inaccuracy of traditional methods, the present invention proposes a security risk relationship extraction method based on multi-view IB, and uses this method to establish a novel and effective model (Multi-View Information Battleneck Relation Extraction, MIBRE). The model provided by this method can automatically extract semantic information from multiple corresponding images, thereby making full use of the semantics of image perspectives to enhance the representation of domain entities. In order to more fully express semantic information, the present invention introduces VggNet to learn the contextual information between sentences. Afterwards, the present invention introduces a heterogeneous graph neural network, which models words and relationship nodes in the same graph structure space to capture contextual semantics. In order to obtain the maximum relevant information between two perspectives, the present invention introduces the information bottleneck theory and formulates it as an information loss function. By enhancing the representation of domain entities and obtaining the maximum relevant information between two perspectives, excellent domain relationship extraction results can be guaranteed. As Figure 1 As shown, the method specifically comprises the following steps:

[0056] 1) Use multi-view semantic fusion strategy to fuse text nodes and image information;

[0057] 1-1) Problem definition: Given multi-view data, let T = {x1, x2, …, x |T|} as the input sentence (x i is a word), |T| represents the length of the sentence; the purpose of domain relation extraction is to identify entity pairs (x i1-i2 ,x j1-j2 ) relationship type, where x i1-i2 、x j1-j2 They represent the spans of the two entities i1-i2 and j1-j2 respectively. A word consists of two characters, so the subscript is the subscript of the two characters. The relationship set is R represents a set of relations, that is, R∈{r1,r2,...,r n}.

[0058] Here is an example from a dataset: "On the left <e1> ditch< / e1> The drainage facilities were not cleared and weeded in time. <e2> Water accumulation< / e2> Serious", the task is to predict the relationship type between entity 1 "ditch" and entity 2 "water accumulation".

[0059] 1-2) Use multi-view semantic fusion strategy to encode and extract security risk knowledge;

[0060] First, the security risk text after word segmentation is used to obtain low-dimensional word vectors using the pre-trained language model BERT, specifically:

[0061] For an input sentence T, the word It is expressed as:

[0062]

[0063] in, d w is the dimension of the word vector; d c is the dimension of the position vector.

[0064] 1-3) In order to enhance the semantic information of domain entities, the present invention adopts the fusion of image perspective semantic information to supplement the insufficient representation in the pre-trained model, specifically:

[0065] For domain entity x i , the image set corresponding to the match is m = {m i1 ,m i2 ,...,m in}, and then use VggNet to obtain the image feature vector m i , directly add the image feature vector to the corresponding entity feature.

[0066] 2) In order to more fully integrate the information between the two perspectives, the present invention uses a deep neural network RetNet to learn the information between text and pictures, so as to use the information between pictures to supplement the deficiencies in the semantic representation of the text.

[0067] 3) Heterogeneous graph spaces are exploited to model word nodes and relation nodes to capture rich contextual semantics with more task-related features.

[0068] In order to better understand the heterogeneous graph of the present invention, the word x i Rewrite as word node The relationship i Rewrite as a relationship node And introduce a message passing mechanism to update the node representation in the following way:

[0069]

[0070] Among them, [·;·] means connecting two vectors into a new representation, represents the neighboring points of the i-th node, W a ,W q ,W k ,W v is the trainable weight; o j The encoding of the relationship, α ij represents the probability value of the attention mechanism, h i ′ represents the updated code, h i Indicates the encoding before updating.

[0071] Then, a gate mechanism is used instead of an activation function to maintain the scale and nonlinearity of each dimension, as shown below:

[0072]

[0073] Among them, W g is the trainable weight, v i is the probability of the gate mechanism, For the final code after passing through the door;

[0074] The above operations of heterogeneous graphs are summarized as follows:

[0075]

[0076] Similarly, the encoding of the relationship node is:

[0077]

[0078] 4) Using the idea of ​​first compressing and then restoring the information bottleneck, the relevant information of the two perspectives is expressed as:

[0079]

[0080] Where ∈ is the standard normal distribution, N is the number of training examples, KL is the Kullback-Leibler divergence, L1 is the loss, I is the mutual information, β is the trade-off parameter, Y is the target encoding, Z is the intermediate encoding, H is the initial encoding, y, z and h are the specific variable observations, Y, Z and H represent the set of variables, p(.) represents the corresponding probability, q(z) is a variational, approximate distribution with p(z), KL[q(z),p(z)]>=0.

[0081] 5) The present invention obtains the corresponding entity representation and sentence representation, uses the max-pooling mechanism to obtain the representation of the corresponding entity, and uses the splicing operation to splice the entity information and sentence information together, and then obtains the final representation of the relationship classification through the feedforward neural network (FFN). r Finally, the present invention inputs the obtained high-level semantics into the softmax classifier to calculate the conditional probability p(r|T,θ) of the security risk relationship to which the entity belongs. The details are as follows:

[0082]

[0083] Where |R| is the number of security risk relationship types, O r It is the final output of the heterogeneous graph neural network, i.e., the prediction results of all security risk relationship types;

[0084] Use cross entropy for classification as follows:

[0085]

[0086] Wherein, L2 represents the loss, n is the number of input sentence sets, and θ is a set of parameters that the model of the present invention needs to learn;

[0087] The total loss of the model is:

[0088] L=L1+L2 (27)

[0089] In model training, the stochastic gradient descent algorithm SGD is used to optimize the objective function.

[0090] The present invention uses deep neural networks to automatically learn the semantic features of security risks. In order to solve the problem of insufficient entity representation in traditional domain relationship extraction, the present invention introduces a multi-perspective method to enhance domain entity representation. When fusing the semantic information of two perspectives, the present invention uses the information bottleneck method to obtain the maximum relevant information between the two perspectives, which is used to represent the fused semantics. By understanding the domain context semantics through heterogeneous graph neural networks, the relationships between entities can be divided into different categories, which is the basis for implementing knowledge-driven security risk identification.

[0091] Simulation experiment and result analysis

[0092] The present invention has five classic indicators to evaluate the performance of the relation extraction model. Specifically:

[0093] Recall rate: It is the proportion of predicted labels in the true label set of statistical samples. The larger its value, the better the performance of the relationship extraction model; it is expressed as:

[0094]

[0095] Where N correct is the number of correctly extracted entity relations, N actual The number of entity relationships extracted for the entity.

[0096] Precision: It is the proportion of the true label of the current sample in the statistical prediction label set. The larger its value, the better the performance of the relationship extraction model; it is expressed as:

[0097]

[0098] Among them, N total is the total number of entity relationships extracted.

[0099] F1: It balances precision and recall. The larger its value, the better the performance of the relation extraction model; it is expressed as:

[0100]

[0101] Among them, Recall is the recall rate, which is the proportion of the predicted label in the true label set of the statistical sample. The larger its value, the better the performance of the relationship extraction model; Precision is the accuracy, which is the proportion of the true label of the current sample in the statistical predicted label set. The larger its value, the better the performance of the relationship extraction model.

[0102] F1 score is an indicator used in statistics to measure the accuracy of a binary classification model. It takes into account both the precision and recall of the classification model. The F1 score can be regarded as a harmonic average of the model's precision and recall, with a maximum value of 1 and a minimum value of 0.

[0103] AUC: It indicates the probability that the probability of a correct sample is greater than the probability of an incorrect sample. The larger its value, the better the performance of the relation extraction model.

[0104] The AUC-ROC curve evaluates model performance from another dimension, and is particularly suitable for binary classification problems. The ROC curve depicts the relationship between the true positive rate (TPR) and the false positive rate (FPR) at different thresholds, and the AUC value is the area under the ROC curve, ranging from 0.5 to 1. The larger the AUC value, the stronger the model's ability to distinguish between positive and negative samples. An ideal model should have an AUC value close to 1. The advantage of the AUC-ROC curve is that it is not affected by changes in the threshold, can fully reflect the overall performance of the model, and is particularly suitable for dealing with class imbalance problems.

[0105] P@N is short for Precision@N, which refers to the accuracy of the first N results for a specific query, taking into account the position factor. For example, if there are 4 relevant documents in the first 5 results of a single search, then P@5 = 4 / 5 = 0.8.

[0106] The present invention uses P@N, AUC and F1 to evaluate the model of the present invention.

[0107] In order to verify that the model of the present invention can automatically acquire safety risk knowledge, the present invention chooses to conduct experiments on real data sets. These data sets are RVSR (Railway Vehicle Safety Risk) and RMSR (Railway Maintenance Safety Risk), which are derived from the safety supervision system of the China Railway Administration. The detailed information of the two data sets is as follows:

[0108] RVSR: The dataset mainly describes the unsafe production conditions of railway sections recorded by security inspectors. It includes 9285 sentences, including 4236 relation instances.

[0109] RMSR: The dataset mainly describes the unsafe production conditions in railway sections recorded by security inspectors. It includes 26,180 sentences, including 10,487 relation instances.

[0110] The present invention took more than 3 months to annotate these data sets, and each sentence was marked by 3 volunteers. There are four safety risk relationships, namely: environmental safety risk, safety reliability of public engineering equipment, personnel operation safety, and safety management effectiveness. The present invention randomly selected 80% of the data as the training data set and 20% of the data as the test data set.

[0111] Comparison of algorithms and experimental settings: In order to verify the effectiveness of the proposed relation extraction model, the experiment of this invention will be compared with three types of relation extraction models, mainly including:

[0112] (1) LGGCN model: It proposes a new knowledge-aware framework graph convolutional network to improve the efficiency and accuracy of long-distance supervised relationship extraction.

[0113] (2) TDGAT model: This model captures the relational links between entities through a dynamic graph attention network and uses the link information to extract more detailed relationship types, effectively reducing error propagation.

[0114] (3) BERT-GMAN model: It significantly improves the performance of relation extraction by introducing key phrase information and adopting a classification feature-aware network.

[0115] (4) PCNN+ATT model: Integrates the self-attention mechanism and segmented CNN to capture rich contextual semantics. This framework further extracts fine-grained features contained in the context.

[0116] (5) PRM-CNN model: This model addresses the problems of word ambiguity and word segmentation errors in Chinese relationship extraction. It proposes a convolutional neural network model based on character input and word meaning reflection mechanism, which effectively improves the performance of Chinese relationship extraction.

[0117] (6) Ensemble BiLSTM model: This model proposes a novel Ensemble BiLSTM method, which combines BERT word embedding and sequence model to effectively improve the accuracy and efficiency of the model.

[0118] (7) SPCNN model: It proposes a new semantic segment convolutional neural network (SPCNN) to address the noise problem in long-distance supervised relationship extraction. It improves the relationship extraction performance by combining the similarity features between entity pairs, and designs an adaptive negative training strategy to effectively reduce the interference of noise labels.

[0119] (8) EMGCE model: A multi-granularity end-to-end causal relationship extraction model is proposed. Through different levels of semantic representation and improved algorithms, cascading errors are effectively avoided and context information is retained.

[0120] In order to explore the effectiveness of enhancing semantics in images, the present invention selects the number of images from {0, 1, 3, 5, 10}. The present invention selects the value of Lagrange multiplier from {0.01, 0.001, 0.0001, 0.00001, 0.000001}. In the selection of learning rate, the present invention selects from {0.0001, 0.0002, 0.0003, 0.0004, 0.0005}, and the best hyperparameters are shown in Table 1. All experiments were conducted on a server equipped with Intel (R) Xeon (R) Silver 4216CPU @ 2.10GHz, 4NVIDIA Tesla V100GPU (each 32GB) and Ubuntu 14.04 operating system.

[0121] Experimental results and analysis: The effectiveness of the proposed model is verified on two real data sets. Tables 2 and 3 show the P@N, AUC and F1 of all models on the two data sets. Figure 2 and Figure 3 The precision-recall curves on RVSR and RMSR are shown.

[0122] Table 1 Hyperparameter settings

[0123]

[0124] The analysis in Table 1 shows that the MIBRE model proposed in the present invention is significantly better than the comparative relation extraction models on two real data sets. Specifically, compared with the LGGCN and TDGAT models, they usually represent the input sentence as a graph, aiming to use graph neural networks to capture more feature information, but do not solve the problem of insufficient representation of domain entities, and the relationship extraction effect is not obvious. Compared with the BERT-GMAN model, this model usually emphasizes phrases to capture task-related features, but does not give prominent weights to entities that are important for relationship extraction but insufficiently represented. Compared with PCNN+ATT and SPCNN, due to the problem of Chinese entity length, segmented PCNN+ATT cannot solve the problem of relation extraction in Chinese well. Finally, compared with PRM-CNN, Ensemble BiLSTM, and EMGCE models, they all aim to improve the ability to understand semantic information. The model of the present invention adopts a multi-perspective method, integrates the semantics of the image perspective, and significantly improves the representation ability of domain entities. At the same time, it also improves the performance of understanding contextual semantics. This observation shows that the introduction of the image perspective in the present invention is conducive to relationship extraction.

[0125] Compared with other methods, the advantages of the MIBRE model of the present invention are mainly reflected in the following two aspects: First, by integrating multi-perspective semantic information, the domain entity semantics is greatly enhanced, the problem of insufficient domain entity semantic representation is solved, and the performance of the relationship extraction model is improved. Second, the maximum relevant information between perspectives is obtained through the information bottleneck method. Compared with the previous method of simply splicing the information of each perspective together, the information within the perspective can be fully utilized, thereby improving the relationship extraction performance of the model.

[0126] In addition, the present invention further studies the impact of different numbers of images on the model. The present invention selects the number of images from {0, 1, 3, 5, 10}, where 0 indicates no embedded image semantics, and the remaining numbers indicate the number of images corresponding to each domain entity.

[0127] Table 2 Performance of different methods on the RVSR dataset

[0128]

[0129] Table 3 Performance of different methods on the RMSR dataset

[0130]

[0131] Table 4 Performance of different numbers of images on the RVSR dataset

[0132]

[0133] Table 5 Performance of different numbers of images on the RMSR dataset

[0134]

[0135] By comparing and analyzing Table 4 and Table 5, the following observations are made: The present invention can find that only one picture can improve the performance of the domain relationship extraction task, which verifies the hypothesis of the present invention that the semantics of the fusion image perspective can enhance the representation of domain entities. As the present invention increases the number of images (from 1 to 10), the performance of the MIBRE model initially increases and then decreases. On these two datasets, the present invention finds that when the number of images is greater than 3, the performance of the MIBRE model begins to decline. This suggests that 3 images are sufficient to enhance the representation, because fusing a large number of images will introduce unexpected irrelevant features.

[0136] Most existing relation extraction models ignore the problem of insufficient representation of domain entities and there are more irrelevant information in the data. To address the above problems, the method of the present invention establishes a security risk relation extraction model MIBRE based on multi-perspective IB, which enhances the semantics of domain entities by fusing multi-perspective semantic information. In order to obtain the maximum relevant information between the two perspectives, an objective function is constructed using the information bottleneck method, which maximizes the retention and restoration of relevant information while compressing the information of the two perspectives. Experiments on two real domain data sets show that the F1 values ​​of MIBRE recognition reach 64.28% and 74.34%, respectively, which are 4.41% and 2.98% higher than the F1 values ​​of the LGGCN model; compared with the TDGAT based on the attention mechanism, the F1 values ​​are increased by 1.89% and 1.53%, respectively. The effectiveness of the algorithm proposed in the present invention in security risk identification is verified. The model uses multi-perspective semantics to enhance entity semantics, and obtains the maximum relevant information between the two perspectives through the information bottleneck theory, which significantly improves the performance of the model.

[0137] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A security risk relationship extraction method based on multi-perspective IB, characterized by: The following steps are involved: 1) Use multi-view semantic fusion strategy to fuse text nodes and image information; 2) Use the deep neural network RetNet to learn the information between text and pictures, so as to use the information between pictures to supplement the deficiencies in the semantic representation of text; 3) Leveraging heterogeneous graph spaces to model word nodes and relation nodes to capture rich contextual semantics with more task-related features; 4) Use information bottlenecks to compress information; 5) Use the max-pooling mechanism to obtain the representation of the corresponding entity, use the concatenation operation to concatenate the entity information and sentence information together, and use the feedforward neural network FFN to obtain the final representation of the relationship classification; and input the obtained high-level semantics into the softmax classifier to calculate the conditional probability of the security risk relationship to which the entity belongs.

2. The method for extracting security risk relationships based on multi-perspective IB according to claim 1 is characterized in that: Step 1) specifically includes: 1-1) Problem definition: Given multi-view data, let T = {x1, x2, …, x |T| } as the input sentence, where x i is a word, |T| represents the length of the sentence; the purpose of domain relation extraction is to identify entity pairs (x i1-i2 ,x j1-j2 ), x i1-i2 、x j1-j2 Respectively represent the span of two entities i1-i2 and j1-j2; the relationship set is 1-2) Use multi-view semantic fusion strategy to encode and extract security risk knowledge; First, the security risk text after word segmentation is used to obtain low-dimensional word vectors using the pre-trained language model BERT, specifically: For an input sentence T, the word It is expressed as: in, d w is the dimension of the word vector; d c is the dimension of the position vector; 1-3) In order to enhance the semantic information of domain entities, the image perspective semantic information is fused to supplement the insufficient representation in the pre-trained model. Specifically: For domain entity x i , the image set corresponding to the match is m = {m i1 ,m i2 ,...,m in }, and then use VggNet to obtain the image feature vector m i , directly add the image feature vector to the corresponding entity feature.

3. The method for extracting security risk relationships based on multi-view IB according to claim 1 is characterized in that: Step 3) specifically includes: The word x i Rewrite as word node The relationship i Rewrite as a relationship node And introduce a message passing mechanism to update the node representation in the following way: Among them, [·;·] means connecting two vectors into a new representation, represents the neighboring points of the i-th node, W a ,W q ,W k ,W v is the trainable weight, o j The encoding of the relationship, α ij represents the probability value of the attention mechanism, h i ′ represents the updated code, h i Indicates the code before updating; Then, a gate mechanism is used instead of an activation function to maintain the scale and nonlinearity of each dimension, as shown below: Among them, W g is the trainable weight, v i is the probability of the gate mechanism, For the final code after passing through the door; The above operations of heterogeneous graphs are summarized as follows: Similarly, the encoding of the relationship node is:

4. The method for extracting security risk relationships based on multi-perspective IB according to claim 1 is characterized in that: In step 4), the idea of ​​first compressing and then restoring the information bottleneck is used to express the relevant information of the two perspectives as follows: Where ∈ is the standard normal distribution, N is the number of training examples, KL is the Kullback-Leibler divergence, L1 is the loss, I is the mutual information, β is the trade-off parameter, Y is the target encoding, Z is the intermediate encoding, H is the initial encoding, y, z and h are the specific variable observations, Y, Z and H represent the set of variables, p(.) represents the corresponding probability, and q(z) is a variational approximation with the same distribution as p(z).

5. The method for extracting security risk relationships based on multi-view IB according to claim 1, characterized in that: In step 5), the conditional probability p(r|T,θ) of the security risk relationship to which the entity belongs is calculated, specifically: Where |R| is the number of security risk relationship types, O r It is the final output of the heterogeneous graph neural network, i.e., the prediction results of all security risk relationship types; Use cross entropy for classification as follows: Wherein, L2 represents the loss, n is the number of input sentence sets, and θ is a set of parameters that the model of the present invention needs to learn; then the total loss of the model is: L=L1+L2 (9) In model training, the stochastic gradient descent algorithm SGD is used to optimize the objective function.

Images