A network threat intelligence relation extraction method based on attention context mapping and relation matching

Abstract

The present invention proposes a network threat intelligence relationship extraction method based on attention context mapping and relationship matching. The method is based on a deep neural network and adopts the following technical solutions: (a) The BERT model pre-trained with large-scale general text data is used to vector encode the input text to obtain a preliminary text vector representation. The BERT model pre-trained with large-scale general text data has acquired some latent semantic information of the text, so the vector representation obtained by encoding the threat intelligence text already contains rich general prior knowledge and latent semantics. (b) The problem of entity semantic abstraction and easy confusion of entity features is alleviated through the attention-based context mapping method. (c) The problem of weak correlation between entities is alleviated through the relationship matching mechanism.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and more specifically, to a network threat intelligence relationship extraction method based on attention context mapping and relationship matching. Background Art

[0002] As cyberspace becomes increasingly complex, cyberattacks such as phishing, spoofing, and ransomware are increasing in number and becoming more complex and diverse. This poses a significant threat to national and corporate cybersecurity. These cyberspaces often contain vast amounts of private data and even state secrets. Once malicious intrusions occur and systems are unable to respond promptly, the consequences can be disastrous. Existing cybersecurity defense technologies can be divided into two parts: 1) Utilizing intrusion detection technology, when systems detect cyberattacks, they promptly respond with defensive measures. 2) Developing and deploying new security solutions based on historical attack records. However, attackers are constantly experimenting with new attack techniques, strategies, and malware, making it difficult to keep up-to-date with relevant defense strategies.

[0003] Knowledge graphs, with their structured storage, efficient search capabilities, and simple, clear visualization, are ideal for tracing cyberattacks and identifying potential threats. Furthermore, with the explosion of big data technology, mining threat information from massive amounts of cyber threat intelligence data and constructing cyber threat intelligence knowledge graphs is becoming a new cybersecurity solution. However, cyber threat intelligence is typically presented as unstructured data and cannot be directly stored as knowledge graphs. Therefore, natural language processing techniques such as named entity recognition, relation extraction, and entity disambiguation are required to convert massive amounts of cyber threat intelligence data from unstructured to structured form. Relation extraction is a key step in this data conversion process.

[0004] Cyber ​​threat intelligence relationship extraction, also known as relationship classification, primarily involves identifying and determining the relationship between two known entities. The primary task is to extract relationships through in-depth mining and analysis of entity information and the context within which the entities reside. Ultimately, the classified relationship triples are stored.

[0005] Compared to general-purpose relation extraction techniques, network threat intelligence relation extraction technology faces the following challenges due to its focus on different entities and relational forms: (a) Entity terms in network threat intelligence are more complex and semantically abstract. (b) Entity density within a single sentence in network threat intelligence is often high, making entity features easily confused and requiring in-depth mining of fine-grained features. (c) Indirect associations often arise between pairs of network threat intelligence entities, resulting in weak correlations between entities and an inability to accurately determine relational categories.

[0006] Therefore, a network threat intelligence relation extraction method based on attention context mapping and relation matching is desired. Summary of the Invention

[0007] In response to the above problems, the present invention proposes a network threat intelligence relationship extraction method based on attention context mapping and relationship matching. The model is based on a deep neural network and adopts the following technical solutions: (a) The BERT model pre-trained with large-scale general text data is used to vector encode the input text to obtain a preliminary text vector representation. The BERT model pre-trained with large-scale general text data has acquired some latent semantic information of the text, so the vector representation obtained by encoding the threat intelligence text already contains rich general prior knowledge and latent semantics. (b) The problem of entity semantic abstraction and easy confusion of entity features is alleviated through the attention-based context mapping method. (c) The problem of weak correlation between entities is alleviated through the relationship matching mechanism.

[0008] Accordingly, according to one aspect of the present application, a network threat intelligence relationship extraction method based on attention context mapping and relationship matching is provided, which includes:

[0009] Construct the encoding layer of the network threat intelligence relationship extraction model;

[0010] Constructing an attention-based context mapping layer for network threat intelligence relation extraction models;

[0011] Construct the relationship matching layer of the network threat intelligence relationship extraction model;

[0012] Use the HACKER dataset to train the model;

[0013] Feed massive amounts of cyber threat intelligence into the model.

[0014] In the above-mentioned network threat intelligence relationship extraction method based on attention context mapping and relationship matching, the encoding layer of the network threat intelligence relationship extraction model is constructed, including:

[0015] Given a sentence Seq as input;

[0016] Each word in the sentence Seq is further decomposed into a series of finer word segmentation units, and the word segmentation set sequence representation E = {e1, e2, ..., e n}, where e i Represents the number of each segmentation, and n represents the segmentation length of the sentence;

[0017] Use the BERT model to encode it and obtain a new sentence representation sequence where d s Indicates si The dimension is as follows:

[0018] S=BERT(E)={s1,s2,…,s n}.

[0019] In the above-mentioned network threat intelligence relationship extraction method based on attention context mapping and relationship matching, an attention-based context mapping layer of the network threat intelligence relationship extraction model is constructed, including: using the pre-trained BERT model to deeply encode the entity and relationship labels to obtain the entity label word embedding matrix Pe and the relationship label word embedding matrix Pr rich in semantic information; taking the entity pairs whose relationships need to be identified as a whole to integrate the potential entity association information, enhance the difference representation between the entity pairs, and obtain a new entity pair representation sequence ef'; combining the relationship label word embedding matrix Pr with the context feature vector ct to obtain a new relationship context feature matrix Cr'; using the attention mechanism to combine the entity pair representation sequence ef' with the relationship context feature matrix Cr' to obtain a relationship context enhanced sequence representation Cp specific to the entity pair.

[0020] In the above-mentioned network threat intelligence relationship extraction method based on attention context mapping and relationship matching, the pre-trained BERT model is used to deeply encode the entity and relationship labels to obtain the entity label word embedding matrix Pe and the relationship label word embedding matrix Pr rich in semantic information, which are used to: obtain the entity and relationship type word embedding, input the entity label set el and the relationship label set rl into the pre-trained BERT model for semantic encoding, and obtain the entity label word embedding matrix And the relationship label word embedding matrix The formula is as follows:

[0021] Pe=BERT(el)={pe1,pe2,...,pe z}

[0022] Pr=BERT(rl)={pr1,pr2,...,pr r}

[0023] Among them, d e is the embedding dimension of entity type label words, z represents the number of entity type label words, d r is the embedding dimension of the relationship type label word, and r is the number of relationship type label words.

[0024] In the above-mentioned network threat intelligence relationship extraction method based on attention context mapping and relationship matching, the entity pairs whose relationships need to be identified are taken as a whole to integrate the potential entity association information, enhance the difference representation between the entity pairs, and obtain a new entity pair representation sequence ef', which is used to: generate the initial head entity span size embedding vector s_h and the tail entity span size embedding vector s_t through a random initialization method. In the multi-layer nonlinear transformation process of the multi-layer perceptron, the span size information is deeply integrated with the original span representation, thereby generating an enhanced entity representation for the two entities that contains both the original text information and rich size feature information. The head entity is represented as eh and the tail entity is represented as et. The formula is as follows:

[0025] eh=[s h ;s_h]·W h +b h

[0026] eh'=tanh(eh)

[0027] et=[s t ;s_t]·W t +b t

[0028] et'=tanh(et)

[0029] in, and is a learnable parameter, d se is the dimension of the span embedding, [;] represents the vector connection operation, tanh represents the hyperbolic tangent activation function, s h is the header entity encoding vector obtained in the previous layer, s t is the tail entity encoding vector obtained in the previous layer;

[0030] The entity pair is regarded as an indivisible whole unit. By introducing a multi-layer perceptron as a fusion mechanism, the sequence representations of the head entity and the tail entity are deeply integrated to capture the interaction and dependency between them, thereby generating a first entity label pair sequence ep'; the head entity label encoding vector and the tail entity label encoding vector are fused to generate a second entity label pair sequence elp'; a third multi-layer perceptron is used to fuse the entity pair sequence representation ep' with the entity label pair sequence representation elp' to generate an entity pair sequence es' enhanced by an entity label pair that contains both sequence information and semantic knowledge; the spacing between the two entities is randomly initialized to obtain an embedded sequence epe; through a fourth multi-layer perceptron, the spacing information of the two entities is fused into the entity pair sequence es' enhanced by the entity label pair to generate the final entity pair representation sequence ef'; the formula is as follows:

[0031] ep=[e h ;e t ]·W ht +b ht

[0032] ep'=tanh(ep)

[0033] elp=[pe h ;pe t ]·W pl +b pl

[0034] elp'=tnnh(elp

[0035] es=[ep';elp']·W el +b el

[0036] es'=tanh(es)

[0037] ef=[es';epe]·W ef +b ef

[0038] ef'=tanh(ef

[0039] in, and is a learnable parameter, d pe is the dimension of entity pair spacing embedding, pe h Encode vector for the head entity label, pe t Encode the tail entity label vector.

[0040] In the above-mentioned network threat intelligence relationship extraction method based on attention context mapping and relationship matching, the relationship label word embedding matrix Pr is combined with the context feature vector ct to obtain a new relationship context feature matrix Cr', which is used to: embed all the relationship label words into the matrix They are respectively combined with the context feature vectors, and through the multi-layer nonlinear transformation of the multi-layer perceptron, the information interaction and fusion between the two are realized to generate a context representation matrix Cr' specific to the relationship. The formula is as follows:

[0041] Cr=[ct;Pr]·W cr +b cr

[0042] Cr'=tanh(Cr)

[0043] in, and are learnable parameters.

[0044] In the above-mentioned network threat intelligence relationship extraction method based on attention context mapping and relationship matching, the entity pair representation sequence ef' is combined with the relationship context feature matrix Cr' using the attention mechanism to obtain the entity pair-specific relationship context enhanced sequence representation Cp, which is used to: measure the importance of each context element for the current entity pair relationship recognition by calculating the attention weight, and accordingly perform weighted adjustment on the relationship-based context sequence representation. After enhancement by the attention mechanism, the entity pair-specific relationship context enhanced sequence representation Cp is obtained, and the formula is as follows:

[0045]

[0046] Cp=score·Cr'

[0047] in, and is a learnable parameter and softmax represents the softmax activation function.

[0048] In the above-mentioned network threat intelligence relationship extraction method based on attention context mapping and relationship matching, the relationship matching layer of the network threat intelligence relationship extraction model is constructed, including:

[0049] The cosine similarity calculation is performed on the relationship label word embedding matrix Pr and the entity-specific relationship context enhanced sequence representation Cp to obtain a similarity score The formula is as follows:

[0050]

[0051] Among them, |||| represents the bi-norm of the vector, and i represents the i-th relationship label;

[0052] The cosine similarity calculation is performed on the relationship label word embedding matrix Pr and the entity pair sequence representation ep' to obtain a similarity score The formula is as follows:

[0053]

[0054] Among them, j represents the jth relationship label

[0055] The similarity score and similarity scores Adaptive weighted fusion is performed to obtain the final relationship classification score sc', and finally the relationship with the largest classification score is output. The formula is as follows:

[0056]

[0057] sc'=softmax(sc

[0058] Among them, α is a trainable adaptive fusion parameter.

[0059] In the above-mentioned network threat intelligence relationship extraction method based on attention context mapping and relationship matching, the HACKER dataset is used to train the model. The HACKER dataset is input into the network threat intelligence relationship extraction model for 50 iterations of training. The parameter settings with the best effect are selected to obtain the optimal network threat intelligence relationship extraction model. The Label Smoothing Loss loss function is selected as the loss function used in training. The formula is as follows:

[0060]

[0061] Where L is the loss function, is the label distribution of the i-th category after smoothing, ε is the smoothing parameter (a small constant), y is the label of the true category, sc i is the probability of the i-th category predicted by the model.

[0062] In the above-mentioned network threat intelligence relationship extraction method based on attention context mapping and relationship matching, massive network threat intelligence is input into the model, which is used to: divide the massive network threat intelligence into sentences, and after completing entity extraction, input the sentence into the network threat intelligence relationship extraction model to obtain the final relationship triples.

[0063] Compared with the existing technology, the present invention proposes a network threat intelligence relationship extraction model based on attention context mapping and relationship matching. The model is designed to address the existing difficulties in relationship extraction technology in the field of network threat intelligence. By deeply mining the potential information of entities and enhancing the interaction between entities and the surrounding text environment, the problems of complex entity words and abstract semantics are alleviated. Using an attention-based context mapping mechanism, entity pairs are directly regarded as a whole to interact with context information, which alleviates the problems of high entity density in single sentences and easy confusion of entity features. Using a relationship matching mechanism, relationship label information is fused and interacted with context information and entity pair information, and at the same time, the classification accuracy is improved by collaboratively processing relationship classification, which alleviates the problem of indirect associations between entity pairs, resulting in weak correlation between entities and inability to accurately judge relationship categories. In the field of network threat intelligence, the recognition effect of the present invention is significantly superior to that of traditional neural network-based relationship extraction models. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is an overall flow chart of a network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to an embodiment of the present application. DETAILED DESCRIPTION

[0065] Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0066] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0067] In addition, numerous specific details are provided in the following detailed description to better illustrate the present application. Those skilled in the art will appreciate that the present application can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present application.

[0068] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0069] Figure 1 The figure shows the overall flow chart of the network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to an embodiment of the present application. Figure 1 As shown, according to the network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to the embodiment of the present application, the specific steps include: constructing an encoding layer of a network threat intelligence relationship extraction model; constructing an attention-based context mapping layer of a network threat intelligence relationship extraction model; constructing a relationship matching layer of a network threat intelligence relationship extraction model; training the model using the HACKER dataset; and inputting massive network threat intelligence into the model.

[0070] In the steps of the embodiment of the present application, a coding layer of a network threat intelligence relationship extraction model is constructed. It should be understood that the core function of the coding layer is to convert the input network threat intelligence text data into a vectorized representation form that can be effectively recognized and processed by the computer. This process is intended to not only retain the structural characteristics of the text data, but also to deeply explore and capture the deep semantic information contained in the sentence and word levels, thereby laying a solid foundation for subsequent threat analysis and identification tasks. The coding layer usually uses a deep learning model, such as BERT (Bidirectional Encoder Representations from Transformers), to capture the contextual information in the text. This is crucial for understanding the semantics of words in sentences. Accordingly, the network threat intelligence sentence sequence is input into the BERT model for encoding to obtain the context feature vector sequence ct and the word segmentation feature matrix S of the sentence.

[0071] Specifically, in one embodiment of the present application, the encoding layer of the network threat intelligence relationship extraction model is constructed, including: given a sentence Seq as input; each word in the sentence Seq is further decomposed into a series of finer word segmentation units, and a word segmentation set sequence representation E = {e1, e2, ..., e m}, where e i Represents the number of each word segmentation, and n represents the word segmentation length of the sentence; it is encoded using the BERT model to obtain a new sentence representation sequence where d s Indicates s i The dimension is as follows:

[0072] S=BERT(E)={s1,s2,…,s n}.

[0073] In the steps of the embodiment of the present application, an attention-based context mapping layer is constructed for the network threat intelligence relationship extraction model. It should be understood that the purpose of the attention-based context mapping layer is to obtain subtle differences between entity pairs, as well as the interactive information between entity pairs and various relationship labels in the corresponding context, to enhance the model's ability to capture the intrinsic connection between entity pairs and relationship labels. Provide rich feature support and powerful decision-making basis for subsequent relationship classification or identification. In network threat intelligence, the relationship between entities may be very subtle and complex. The attention mechanism can help the model identify and emphasize the specific connection between entities. Compared with traditional recurrent neural networks (RNNs), attention mechanisms usually have fewer parameters, which can reduce the computational complexity and resource consumption of the model. In the relationship extraction task, it is crucial to correctly identify the exact relationship between entities. The attention mechanism can improve the model's ability to identify these relationships, especially when the relationship is not obvious or ambiguous. When processing long texts or complex sentence structures, the attention mechanism can effectively capture long-distance dependencies, which is particularly important for understanding long-distance entity relationships in texts.

[0074] Specifically, in one embodiment of the present application, an attention-based context mapping layer of a network threat intelligence relationship extraction model is constructed, including: using a pre-trained BERT model to deeply encode entity and relationship labels to obtain an entity label word embedding matrix Pe and a relationship label word embedding matrix Pr that are rich in semantic information; taking entity pairs whose relationships need to be identified as a whole to integrate potential entity association information, enhance the difference representation between entity pairs, and obtain a new entity pair representation sequence ef'; combining the relationship label word embedding matrix Pr with the context feature vector ct to obtain a new relationship context feature matrix Cr'; using the attention mechanism to combine the entity pair representation sequence ef' with the relationship context feature matrix Cr' to obtain a relationship context enhanced sequence representation Cp specific to the entity pair.

[0075] Accordingly, in a specific example of the present application, a pre-trained BERT model is used to deeply encode entity and relationship labels to obtain an entity label word embedding matrix pe and a relationship label word embedding matrix Pr that are rich in semantic information. It should be understood that the BERT model can capture rich semantic information and contextual relationships by training on a large-scale corpus. This enables it to provide deep semantic encoding for entity and relationship labels and enhance the model's ability to understand text. The pre-trained BERT model has been trained on a wide range of text data and has good generalization capabilities. This means that it can handle various types of text, including specific terms and expressions in cyber threat intelligence. Specifically, it includes: obtaining entity and relationship type word embeddings, inputting the entity label set el and the relationship label set rl into the pre-trained BERT model for semantic encoding, and obtaining the entity label word embedding matrix And the relationship label word embedding matrix The formula is as follows:

[0076] Pe=BERT(el)={pe1,pe2,…,pe z}

[0077] Pr=BERT(rl)={pr1,pr2,…,pr r}

[0078] Among them, d e is the embedding dimension of entity type label words, z represents the number of entity type label words, d r is the embedding dimension of the relationship type label word, and r is the number of relationship type label words.

[0079] Accordingly, in a specific example of the present application, the entity pairs whose relationships need to be identified are used as a whole to integrate potential entity association information, enhance the difference representation between entity pairs, and obtain a new entity pair representation sequence ef'. It should be understood that in order to enrich and enhance the expressive power of the entity span representation sequence, the multi-layer perceptron is used to seamlessly integrate the span size information of the entity into its span representation. Specifically, the initial head entity span size embedding vector s_h and the tail entity span size embedding vector s_t are generated by a random initialization method. In the multi-layer nonlinear transformation process of the multi-layer perceptron, the span size information is deeply integrated with the original span representation, thereby generating an enhanced entity representation for the two entities that contains both the original text information and rich size feature information, where the head entity is represented as eh and the tail entity is represented as et. The formula is as follows:

[0080] eh=[s h ;s_h]·W h +b h

[0081] eh'=tanh(eh)

[0082] et=[s t ;s_t]·W t +b t

[0083] et'=tanh(et)

[0084] in, and is a learnable parameter, d se is the dimension of the span embedding, [;] represents the vector connection operation, tanh represents the hyperbolic tangent activation function, s h is the header entity encoding vector obtained in the previous layer, s tis the tail entity encoding vector obtained in the previous layer.

[0085] Furthermore, to deeply explore the intrinsic connection between two entities and enhance the model's ability to discern fine-grained differences between entity pairs, a holistic fusion and enhancement strategy is adopted. Specifically, we first treat the entity pair as an indivisible unit and, by introducing a multi-layer perceptron (MLP) as a fusion mechanism, deeply integrate the sequence representations of the head and tail entities to capture their interactions and dependencies, thereby generating a new entity pair sequence representation, ep'. This process effectively integrates the feature information of the head and tail entities, laying the foundation for subsequent analysis. Next, to further enhance the semantic richness of the entity pair representation, we again utilize an MLP to fuse the head and tail entity labels, aiming to incorporate the semantic knowledge embodied in the label information into the entity pair representation, generating a new entity label pair sequence representation, elp'. This step not only preserves the categorical information of the labels but also enhances the discriminative power of the entity pair representation through the fusion process. Subsequently, to combine the advantages of the entity pair sequence representation, ep', and the entity label pair sequence representation, elp', we employ a third MLP to fuse the entity pair sequence representation, ep', with the entity label pair sequence representation, elp'. This fusion process aims to combine the sequential features of entity pairs with the semantic features of label pairs to generate an entity pair sequence representation es' enhanced by entity label pairs that contains both sequential information and rich semantic knowledge. This representation not only strengthens the feature expression within the entity pair, but also improves the generalization ability of the model in complex scenarios. Finally, in order to more comprehensively characterize the spatial relationship between entity pairs, spacing information is introduced as an additional feature input, and the spacing between the two entities is randomly initialized to obtain the embedded sequence representation epe. Through the fourth multi-layer perceptron, the spacing information of the two entities is fused into the entity pair sequence representation es' enhanced by the entity label pair to generate the final entity pair representation sequence ef'. This process not only enhances the spatial sensitivity of the entity pair representation, but also enables the model to more accurately capture the subtle differences and intrinsic connections between entity pairs, providing more accurate and rich feature support for subsequent relationship extraction. The formula is as follows:

[0086] ep=[e h ;e t ]·W ht +b ht

[0087] ep'=tanh(ep)

[0088] elp=[pe h ;pe t ]·W pl +b pl

[0089] elp'=tanh(elp)

[0090] es=[ep';elp']·W el +b el

[0091] es'=tanh(es)

[0092] ef=[es';epe]·W ef +b ef

[0093] ef'=tanh(ef

[0094] in, and is a learnable parameter, d pe is the dimension of entity pair spacing embedding, pe h Encode vector for the head entity label, pe t Encode the tail entity label vector.

[0095] Accordingly, in a specific example of this application, the relationship label word embedding matrix Pr is combined with the context feature vector ct to obtain a new relationship context feature matrix Cr'. It should be understood that in order to construct a context representation specific to the relationship type, a multi-layer perceptron is used as a fusion mechanism to deeply integrate the relationship label information into the context feature vector. Specifically, all the relationship label words are embedded in the matrix They are respectively combined with the context feature vectors, and through the multi-layer nonlinear transformation of the multi-layer perceptron, the information interaction and fusion between the two are realized to generate a context representation matrix Cr' specific to the relationship. The formula is as follows:

[0096] Cr=[ct;Pr]·W cr +b cr

[0097] Cr'=tanh(Cr)

[0098] in, and are learnable parameters.

[0099] Accordingly, in a specific example of the present application, the entity pair representation sequence ef' is combined with the relationship context feature matrix Cr' using the attention mechanism to obtain the relationship context enhanced sequence representation Cp specific to the entity pair. It should be understood that in order to enhance the interaction between the entity pair and the relationship-based context, the attention mechanism is used to guide the model to adaptively focus on the relationship context information that is more closely related to the entity pair. Specifically, the importance of each context element to the current entity pair relationship recognition is measured by calculating the attention weight, and the relationship-based context sequence representation is weighted and adjusted accordingly. After the enhancement of the attention mechanism, the relationship context enhanced sequence representation Cp specific to the entity pair is obtained, and the formula is as follows:

[0100]

[0101] Cp=score·Cr'

[0102] in, and is a learnable parameter and softmax represents the softmax activation function.

[0103] In the steps of the embodiment of the present application, a relationship matching layer of the network threat intelligence relationship extraction model is constructed. It should be understood that the purpose of the relationship matching layer is to combine the features obtained in the above steps, maximize the accuracy of the model in extracting relationships, and finally classify the relationships between entity pairs. Specifically, it includes: in order to achieve a quantitative evaluation of the similarity between the relationship label and the entity pair relationship context, and provide a basis for the final relationship classification, the cosine similarity calculation is performed on the relationship label word embedding matrix Pr and the entity pair-specific relationship context enhanced sequence representation Cp to obtain a similarity score. The formula is as follows:

[0104]

[0105] Among them, |||| represents the bi-norm of the vector, and i represents the i-th relationship label;

[0106] In order to achieve a quantitative evaluation of the similarity between the relationship label and the entity pair and provide a basis for the final relationship classification, the cosine similarity calculation is performed on the relationship label word embedding matrix Pr and the entity pair sequence representation ep' to obtain a similarity score The formula is as follows:

[0107]

[0108] Among them, j represents the jth relationship label

[0109] The similarity score and similarity scores Adaptive weighted fusion is performed to obtain the final relationship classification score sc', and finally the relationship with the largest classification score is output. The formula is as follows:

[0110]

[0111] sc'=softmax(sc)

[0112] Among them, α is a trainable adaptive fusion parameter.

[0113] It should be understood that the purpose of the relationship matching layer is to combine the features obtained in the above steps to maximize the accuracy of the model in extracting relationships and to finally classify the relationships between entity pairs. The specific steps are as follows:

[0114] In the steps of the embodiment of this application, the HACKER dataset is used to train the model, which is used to: input the HACKER dataset into the network threat intelligence relationship extraction model for 50 iterations of training, select the parameter settings that achieve the best results, and obtain the optimal network threat intelligence relationship extraction model. To alleviate the overfitting problem and the negative impact of noisy labels, the Label Smoothing Loss loss function is selected as the loss function used in training. The formula is as follows:

[0115]

[0116] Where L is the loss function, is the label distribution of the i-th category after smoothing, ε is the smoothing parameter (a small constant), y is the label of the true category, sc i is the probability of the i-th category predicted by the model.

[0117] In the steps of the embodiment of the present application, massive network threat intelligence is input into a model for: dividing the massive network threat intelligence into sentences, and after completing entity extraction, inputting the network threat intelligence relationship extraction model to obtain the final relationship triples.

[0118] In summary, the network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to the embodiment of the present application is designed to address the existing difficulties in relationship extraction technology in the field of network threat intelligence. By deeply mining the potential information of entities and enhancing the interaction between entities and the surrounding text environment, the problems of complex entity words and abstract semantics are alleviated. Using an attention-based context mapping mechanism, entity pairs are directly regarded as a whole to interact with context information, which alleviates the problem of high entity density in single sentences and easy confusion of entity features. Using a relationship matching mechanism, relationship label information is fused and interacted with context information and entity pair information, and at the same time, the classification accuracy is improved by collaboratively processing relationship classification, which alleviates the problem that indirect associations are easy to occur between entity pairs, resulting in weak correlation between entities and inability to accurately judge relationship categories. In the field of network threat intelligence, the recognition effect of the present invention is significantly superior to that of traditional neural network-based relationship extraction models.

[0119] The above describes in detail the preferred embodiments of the present invention. It should be understood that numerous modifications and variations based on the concepts of the present invention are possible by those skilled in the art without inventive effort. Therefore, any technical solution that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A network threat intelligence relationship extraction method based on attention context mapping and relationship matching, characterized in that: include: Construct the encoding layer of the network threat intelligence relationship extraction model; Constructing an attention-based context mapping layer for network threat intelligence relation extraction models; Construct the relationship matching layer of the network threat intelligence relationship extraction model; Use the HACKER dataset to train the model; Feeding massive amounts of cyber threat intelligence into the model; Construct the encoding layer of the network threat intelligence relationship extraction model, including: Given a sentence As input; The sentence Each word in is further decomposed into a series of finer word segmentation units to obtain the word segmentation set sequence representation ,in Indicates the number of each word, Indicates the length of the sentence segment; Use the BERT model to encode it and obtain a new sentence representation sequence ,in express The dimension is as follows: ； Constructing an attention-based context mapping layer for the network threat intelligence relationship extraction model, including: Use the pre-trained BERT model to deeply encode entity and relationship labels to obtain entity label word embedding matrix rich in semantic information and relation label word embedding matrix ; Taking the entity pairs whose relationships need to be identified as a whole, the potential entity association information is integrated to enhance the difference representation between entity pairs and obtain a new entity pair representation sequence. '; Embed the relationship label word into the matrix and context feature vector Combine to get a new relation context feature matrix '; Use the attention mechanism to represent the entity pair sequence 'With the relationship context feature matrix Combined to obtain entity pair-specific relational context enhanced sequence representation ; Constructing the relationship matching layer of the network threat intelligence relationship extraction model, including: Embed the relationship label word matrix and the entity-pair-specific relational context-enhanced sequence representation Perform cosine similarity calculation to get similarity score , the formula is as follows: in, represents the two-norm of the vector, Indicates the Relationship tags; Embed the relationship label word matrix and entity pair sequence representation Perform cosine similarity calculation to get similarity score , the formula is as follows: in, Indicates the Relationship tags The similarity score and similarity scores Perform adaptive weighted fusion to obtain the final relationship classification score ', and finally output the relationship with the largest classification score. The formula is as follows: in, is a trainable adaptive fusion parameter.

2. A network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to claim 1, characterized in that: Use the pre-trained BERT model to deeply encode entity and relationship labels to obtain entity label word embedding matrix rich in semantic information and relation label word embedding matrix , used to: obtain entity and relationship type word embedding, entity label set and relationship label sets Input the pre-trained BERT model for semantic encoding to obtain the entity label word embedding matrix And the relationship label word embedding matrix , the formula is as follows: in, is the entity type label word embedding dimension, Represents the number of entity type label words, is the relationship type label word embedding dimension, is the number of relationship type label words.

3. A network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to claim 2, characterized in that: Taking the entity pairs whose relationships need to be identified as a whole, the potential entity association information is integrated to enhance the difference representation between entity pairs and obtain a new entity pair representation sequence. , used to generate the initial head entity span size embedding vector through random initialization method and tail entity span size embedding vector In the multi-layer nonlinear transformation process of the multi-layer perceptron, the span size information is deeply integrated with the original span representation, thereby generating an enhanced entity representation for the two entities that contains both the original text information and rich size feature information, where the head entity is represented as , the tail entity is represented as , the formula is as follows: in, 、 、 and is a learnable parameter, is the dimension of the embedding span, Represents a vector concatenation operation, represents the hyperbolic tangent activation function, is the header entity encoding vector obtained in the previous layer, is the tail entity encoding vector obtained in the previous layer; The entity pair is regarded as an indivisible whole unit. By introducing a multi-layer perceptron as a fusion mechanism, the sequence representations of the head entity and the tail entity are deeply integrated to capture the interaction and dependency between them, thereby generating the first entity pair sequence representation. '; The head entity label encoding vector and the tail entity label encoding vector are fused to generate the second entity pair sequence representation ; Use the third multi-layer perceptron to represent the entity pair sequence Sequence representation with entity labels 'Fuse to generate an entity pair sequence that contains both sequence information and semantic knowledge ; Randomly initialize the distance between two entities to obtain the embedding sequence ; Through the fourth multi-layer perceptron, the distance information of the two entities is integrated into the entity pair sequence enhanced by the entity label pair ', to generate the final entity pair representation sequence ';The formula is as follows: in, 、 、 、 、 、 、 and is a learnable parameter, is the dimension of entity pair spacing embedding, Encode the vector for the head entity label, Encode the tail entity label vector.

4. A network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to claim 3, characterized in that: Embed the relationship label word into the matrix and context feature vector Combine to get a new relation context feature matrix , used to: embed all relationship label words into the matrix Combined with the context feature vector respectively, through the multi-layer nonlinear transformation of the multi-layer perceptron, the information interaction and fusion between the two are realized to generate a context representation matrix specific to the relationship , the formula is as follows: in, and are learnable parameters.

5. A network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to claim 4, characterized in that: Use the attention mechanism to represent the entity pair sequence The relationship context feature matrix 'Combined to obtain entity pair-specific relational context enhanced sequence representation , which is used to: measure the importance of each context element for the current entity pair relationship recognition by calculating the attention weight, and accordingly make weighted adjustments to the relationship-based context sequence representation. After the enhancement of the attention mechanism, the entity pair-specific relationship context enhanced sequence representation is obtained. , the formula is as follows: in, and is a learnable parameter, softmax represent softmax Activation function.

6. The network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to claim 1 is characterized in that: The HACKER dataset was used to train the model. The HACKER dataset was input into the network threat intelligence relationship extraction model for 50 iterations of training. The parameter settings that achieved the best results were selected to obtain the optimal network threat intelligence relationship extraction model. The Label Smoothing Loss loss function was selected as the loss function used in training. The formula is as follows: in, L is the loss function, For the The label distribution after smoothing of categories, is the smoothing parameter, is the label of the true category, The model predicts the The probability of a category.

7. A network threat intelligence relationship extraction method based on attention context mapping and relationship matching according to claim 6, characterized in that: The massive network threat intelligence is input into the model, which is used to: divide the massive network threat intelligence into sentences, and after completing the entity extraction, input the network threat intelligence relationship extraction model to obtain the final relationship triples.

Images