Entity recognition method

Through the combination of SecureBERT and Cross-BiLSTM, semantic features of threat intelligence text are extracted and global feature representation fusion is carried out, which solves the problem of insufficient accuracy of existing models when dealing with complex context dependencies and new threat entity recognition, and significantly improves threats The effect of intelligence naming entity recognition.

CN119067120APending Publication Date: 2024-12-03BEIJING INFORMATION SCI & TECH UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411193791.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing threat intelligence named entity identification models have difficulty accurately capturing entity boundaries and implications when processing data with special context dependencies, especially in the face of complex attack chain descriptions and new threat entities, with insufficient accuracy and robustness. .

Method used

SecureBERT pre-trained language model is used to extract semantic feature matrix, and these features are processed through Cross-BiLSTM enhancement model, combined with global sentence representation and weight matrix for fusion, and the fused global feature representation is obtained to improve the accuracy of entity recognition.

Benefits of technology

Through the combination of SecureBERT and Cross-BiLSTM, the model can better understand the global context information in threat intelligence text, improving the accuracy and robustness of entity recognition, especially when dealing with long text and new threat entities. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067120A_ABST
    Figure CN119067120A_ABST
Patent Text Reader

Abstract

The invention provides an entity recognition method, which comprises the following steps of: processing a threat intelligence sequence by using a pre-training language model to obtain a semantic feature matrix corresponding to the threat intelligence sequence; wherein the semantic feature matrix comprises a plurality of elements; processing the semantic feature matrixes by adopting an enhancement model to obtain a final hidden state of each element in each semantic feature matrix; and obtaining a global sentence representation; fusing the final hidden state of each element with the global sentence representation to obtain a global vector corresponding to each element; obtaining different weights of the global vector according to the operation results of the global vector of each element and different weight matrixes; performing operation fusion on the final hidden state of each element and the global sentence representation according to different weights of the global vector to obtain a fused global feature representation; obtaining a label sequence probability based on the fused global feature representation; and outputting the entity type with the maximum probability value.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer security technology, and in particular to an entity recognition method. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] Threat Intelligence is an important part of modern network security, which mainly involves collecting, analyzing and interpreting information about potential threats and attacks. Its purpose is to help organizations identify, prevent and respond to network security threats by understanding threat attacks and their behavior patterns.

[0004] At present, with the increase in the number of security vulnerabilities, the rising proportion of high-risk vulnerabilities, and the emergence of new vulnerabilities and attack methods, the severity of the network security environment has intensified. Traditional analysis technology for multi-source threat intelligence information relies on a lot of manual work, with low accuracy and time-consuming, which may lead to missed opportunities to analyze attacker intentions and defense options. Therefore, how to accurately and efficiently obtain key entity information from threat intelligence data is an urgent problem that needs to be solved in the current threat intelligence field.

[0005] Named entity recognition (NER) is an important basic task in natural language processing, and plays an important supporting role in public opinion analysis, automatic question answering, knowledge graphs and other tasks. Named entity recognition in the field of threat intelligence aims to identify a variety of threat entity names and their corresponding types from a large amount of unstructured threat intelligence text, such as attack activities, malware, vulnerabilities and hacker organizations, to help researchers conduct security analysis and evaluation, so as to carry out targeted defense against threat attacks such as vulnerabilities and Trojans.

[0006] In recent years, deep learning derived from neural networks has been widely used in important application scenarios. Long short-term memory networks (LSTM) and bidirectional long short-term memory networks (BiLSTM) have achieved good results in named entity recognition in the field of threat intelligence. However, when BiLSTM processes threat intelligence data with special contextual dependencies, it is affected by the XOR restriction problem, that is, when there are multiple conflicting or interdependent entities in the input sequence, it is difficult for the model to accurately capture and distinguish the boundaries and meanings of these entities. To accurately understand the meaning of threat intelligence text sentences and determine the entity category, it is necessary to consider the context or even the full text context. In the field of threat intelligence, this special contextual dependency is particularly common because threat intelligence usually contains a large number of professional terms, abbreviations, technical details, and complex attack chain descriptions. For example, in the text "Mirai launched DDoS attacks on multiple companies including Twitter, resulting in a sudden increase in traffic peaks", "Mirai" is a malware that is used to launch "DDoS attacks" on targets such as "Twitter". This attack behavior caused a sudden increase in "traffic peaks". This complex dependency requires the model to accurately capture it on the basis of understanding the entire attack activity. BiLSTM mainly relies on the adjacent context information to construct the feature representation of the current moment, and it is difficult to handle long-distance dependencies in the text and obtain more extensive global context information. At the same time, the continuous emergence of new threat entities also poses new challenges to entity recognition models. When the existing threat intelligence named entity recognition model processes new words or unknown entity words that have not been encountered in the training phase, there are significant differences between the features extracted from unknown entities and similar known entities, causing the model to often incorrectly identify the entity type of unknown words, making the NER task more complicated.

[0007] Early research on named entity recognition of threat intelligence mainly adopted entity recognition methods based on rules and dictionaries, relying on experts in the field of network security to build rule templates. Zhang Shuzhuang et al. analyzed the matching method of finite state automata, used regular expressions to enhance the description of protocols and attack features in threat intelligence, and used this to identify the types of threat attacks; while Balduccini et al. combined regular expressions with threat intelligence concept ontology to extract and parse threat intelligence entities from data sources such as log files. Methods based on statistical machine learning can automatically extract and learn threat-related features from raw data, reducing the manual construction of rule dictionaries. Mulwad et al. used support vector machines (SVMs) and supervised learning classification to extract descriptions of specific vulnerabilities and attack entities from training samples; Joshi et al. used conditional random fields (CRFs) to identify threat intelligence knowledge in text and represented it through network security domain ontology. Such methods increase the complexity and cost of the work to a certain extent, thereby limiting the robustness and generalization of the model.

[0008] In recent years, with the development of deep learning technology, entity recognition methods based on neural networks have received widespread attention. Qin Ya et al. used artificial feature templates to extract local context features, and used CNN and BiLSTM neural networks as the basic framework to extract information features of threat intelligence entities, and further used CRF as a label decoder to obtain the association information between labels; Li et al. proposed a neural network model based on self-attention to identify threat intelligence entities. On the basis of the existing BiLSTM-CRF model, more context information related to the words in the current sentence was extracted through the self-attention mechanism. Due to the lack of large-scale annotated corpus in threat intelligence, most studies train threat intelligence data through word embedding models such as Word2Vec and Glove. For the problem that the newly emerging unknown entity words in threat intelligence lack corresponding representation in the word embedding model, Zhang Han et al. used a generative adversarial network in the BiLSTM-Attention-CRF model and solved the problem of data feature distribution bias in the field of threat intelligence by calculating the correlation between each word and the document-level global feature vector. Based on the research on Transformer structure, Evangelatos et al. added a linear classification layer on the basis of pre-trained language models such as BERT, conducted experiments on threat intelligence (DNRTI) datasets and achieved good recognition results. The NER model of the above threat intelligence is mainly based on neural network structures such as BiLSTM as feature extractors. Although the bidirectional long short-term memory network can effectively capture the forward and backward information of the sequence, it is limited by the XOR problem when processing threat intelligence data with special context dependencies. In addition, when a large number of unknown threat entity words appear, it is difficult to obtain the global context information of the entire sequence. Especially when processing long texts, the advantages of the network cannot be effectively utilized. Summary of the invention

[0009] In order to solve at least one of the above technical problems, the present disclosure provides an entity recognition method.

[0010] The present disclosure provides an entity recognition method, which includes:

[0011] Obtaining a threat intelligence sequence including a threat intelligence entity;

[0012] Processing the threat intelligence sequence using the SecureBERT pre-trained language model to obtain a semantic feature matrix corresponding to the threat intelligence sequence; wherein the semantic feature matrix includes a plurality of elements;

[0013] The semantic feature matrix is ​​processed using a Cross-BiLSTM enhancement model to obtain a final hidden state of each element in each semantic feature matrix; and a global sentence representation is obtained;

[0014] The final hidden state of each element is fused with the global sentence representation to obtain a global vector corresponding to each element;

[0015] According to the operation results of the global vector of each element and different weight matrices, different weights of the global vector are obtained;

[0016] The final hidden state of each element is fused with the global sentence representation according to the different weights of the global vector to obtain the fused global feature representation;

[0017] Based on the fused global feature representation, the label sequence probability is obtained, and the entity type with the maximum probability value is output.

[0018] According to an entity recognition method of an embodiment of the present disclosure, obtaining the final hidden state of each element in each semantic feature matrix includes:

[0019] According to the current element, the first layer of forward hidden state representation is obtained by forward LSTM calculation and the second layer of backward hidden state representation is obtained by reverse LSTM calculation;

[0020] The forward hidden state representation of the first layer and the backward hidden state representation of the second layer are concatenated as the input of the Cross-BiLSTM enhanced model, and the forward hidden state representation of the second layer is obtained by forward LSTM calculation, and the backward hidden state representation of the second layer is obtained by backward LSTM calculation;

[0021] The second layer forward hidden state representation and the backward hidden state representation are concatenated to obtain the final hidden state of the current element.

[0022] According to an entity recognition method of an embodiment of the present disclosure, the global sentence representation in, The second layer backward hidden state representation of the element at the first element position; is the second layer forward hidden state representation of the element at the nth element position, where n is the number of elements.

[0023] According to an entity recognition method of an embodiment of the present disclosure, the final hidden state of each element is fused with the global sentence representation to obtain a global vector corresponding to each element, including:

[0024] O t =G||H t ;

[0025] Among them, O t is the global vector of the element at element position t; G is the global sentence representation; H t is the final hidden state representation of the element at element position t.

[0026] According to an entity recognition method of an embodiment of the present disclosure, different weights of the global vector are obtained according to the operation results of the global vector of each element and different weight matrices, including:

[0027]

[0028]

[0029] Among them, W H is the first weight matrix; W G is the second weight matrix; b H is the first bias term; b G is the second bias term; sigmoid() represents the function; where the different weights of the global vector include: is the first weight associated with the final hidden state; is the second weight associated with the global sentence representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0031] Figure 1 is a flowchart of an entity recognition method according to one embodiment of the present disclosure.

[0032] Figure 2 2 is a diagram of the SecureBERT architecture for the MLM pre-training task according to an embodiment of the present disclosure.

[0033] Figure 3 It is a schematic diagram of the model structure of an entity recognition method according to an embodiment of the present disclosure.

[0034] Figure 4 4 is a comparative experimental table of hidden state sizes according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0036] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0037] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0038] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0039] The terms used herein are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0040] Figure 1 is a flowchart of an entity recognition method according to one embodiment of the present disclosure.

[0041] like Figure 1 As shown, the entity recognition method of the present disclosure may include:

[0042] S110, obtaining a threat intelligence sequence including a threat intelligence entity;

[0043] S112. Process the threat intelligence sequence using the SecureBERT pre-trained language model to obtain a semantic feature matrix corresponding to the threat intelligence sequence; wherein the semantic feature matrix includes a plurality of elements;

[0044] S114, using the Cross-BiLSTM enhancement model to process the semantic feature matrix, obtain the final hidden state of each element in each semantic feature matrix; and obtain the global sentence representation;

[0045] S116, fusing the final hidden state of each element with the global sentence representation to obtain a global vector corresponding to each element;

[0046] S118, obtaining different weights of the global vector according to the operation results of the global vector of each element and different weight matrices;

[0047] S120, performing calculation fusion on the final hidden state of each element and the global sentence representation according to different weights of the global vector to obtain a fused global feature representation;

[0048] S122. Based on the fused global feature representation, obtain the label sequence probability; and output the entity type with the maximum probability value.

[0049] The entity recognition method of the present invention first uses the language model SecureBERT pre-trained on a large-scale network security text corpus to extract the semantic features of the threat intelligence sequence; then extracts the hidden layer feature vector representation through the trained Cross-BiLSTM enhanced model, and finally combines the semantic features of the threat intelligence sequence with the vector hidden features, and decodes the fused global feature representation to help the model better understand the global context information in the sequence, thereby improving the accuracy of the threat intelligence entity recognition task.

[0050] The specific steps of the entity recognition method disclosed in the present invention will be described in detail below.

[0051] S110. Obtain a threat intelligence sequence including a threat intelligence entity.

[0052] The threat intelligence sequence disclosed in the present invention may be one paragraph or multiple paragraphs; and the threat intelligence sequence may be expressed in languages ​​such as English, or in languages ​​such as Chinese, or in a mixture of Chinese and English, and the present invention is not limited to this.

[0053] The following is an example of a threat intelligence sequence of a content. Among them, one or more sentences may be included in the threat intelligence sequence. Accordingly, the sentences of the threat intelligence sequence may include at least one threat intelligence entity, and of course, the threat intelligence entity may not be included; in other words, at least one sentence of the threat intelligence sequence includes a threat intelligence entity, and the method disclosed in the present invention can identify the threat intelligence entity and confirm the label category to which the threat intelligence entity belongs.

[0054] In one embodiment, the tag categories can be predefined. For example, the DNRTI dataset contains 13 tag categories, namely hacker organization (HackOrg), attack (OffAct), sample file (SamFile), security team (SecTeam), tool (Tool), time (Time), purpose (Purp), area (Area), industry (Idus), organization (Org), way (Way), vulnerability (Exp), and features (Features).

[0055] S112. Use the SecureBERT pre-trained language model to process the threat intelligence sequence to obtain a semantic feature matrix corresponding to the threat intelligence sequence; wherein the semantic feature matrix includes multiple elements.

[0056] New entities or terms (unknown words) in threat intelligence lack corresponding representation in pre-trained models. Traditional general language models have limitations when processing threat intelligence text. The prediction results provided by general language models such as BERT or RoBERT may be ambiguous, and it is difficult to learn the correlation features between sentences and sentence pairs, which cannot be applied to the field of threat intelligence. For example, in the "botnet attack <mask>In a simple sentence like "Attack", SecureBERT is able to understand the context and correctly predict the masked word as "DDoS".

[0057] Therefore, this disclosure adopts SecureBERT which is pre-trained with MLM on a large-scale cybersecurity text corpus. The SecureBERT network architecture is as follows: Figure 2 As shown in the figure, the MLM task uses the MASK tag to randomly mask 15% of the words, and trains the SecureBERT model to predict the vectors of these masked words. Since the model is not sure which words will be masked, it can only predict the masked words based on contextual information. At the same time, the byte pair encoding (BPE) method is used to add and construct specific terms and conceptual vocabulary in the field of threat intelligence, so as to effectively learn the semantic knowledge of threat intelligence, improve the feature extraction ability of the model, and reduce the feature vector difference between unknown words and entities of the same type. In addition, SecureBERT improves the efficiency of model training by re-optimizing the training process by introducing random noise into the initial weights.

[0058] The threat intelligence sequence is used as the input of the trained SecureBERT language model to obtain a semantic feature matrix corresponding to the threat intelligence sequence; in the present disclosure, the semantic feature matrix may include multiple elements, each element may include multiple vectors, and these vectors may include word vectors and sentence vectors (or segment vectors), etc. The word vectors and sentence vectors will be used as the input of the following Cross-BiLSTM enhanced model.

[0059] S114. Use the Cross-BiLSTM enhancement model to process the semantic feature matrix to obtain the final hidden state of each element in each semantic feature matrix; and obtain the global sentence representation.

[0060] The long short-term memory network (LSTM) introduces forget gate, input gate and output gate structures to control the information state of the neural unit. It selectively ignores certain information in the input sequence to learn and capture important features of the sequence. Therefore, LSTM can solve the gradient explosion and gradient disappearance problems of traditional recurrent neural networks. Based on LSTM, BiLSTM uses a bidirectional recursive neural network structure to add a reverse calculation process. It processes forward and reverse inputs at each time step, and collaboratively generates feature representations containing bidirectional context information from the forward hidden layer vector and the reverse hidden layer vector.

[0061] However, the BiLSTM model has an XOR limitation when dealing with sequence labeling tasks in the field of threat intelligence. Although BiLSTM can capture bidirectional contextual information, each word (token) is still represented using its local hidden state, which cannot fully capture contextual information across different time steps, especially threat intelligence data with special contextual dependencies. To solve this problem, the present disclosure introduces the Cross-BiLSTM enhanced model to enhance the modeling ability of cross-context information. Compared with BiLSTM, in the Cross-BiLSTM model, the hidden states of the forward and backward sequences are interlaced and fused between the stacked LSTM layers, and the cross-context information of the entire sequence is calculated through feedforward transfer and accumulation operations.

[0062] Specifically, for a given threat intelligence sequence (input sequence) S = {w1,w2,…,w n }; The semantic feature matrix generated by the trained SecureBERT language model is expressed as:

[0063] Z={z1,z2,…,z n };

[0064] Among them, n represents the length of the input sentence; that is, the threat intelligence sequence has a total of n sentences; that is, the element of the z1 semantic feature matrix, and z1 is the element in the semantic feature matrix corresponding to the sentence w1.

[0065] For one element position t and its corresponding element (or current element), the final hidden state of the current element can be obtained by the following method, where t is greater than or equal to 1 and less than or equal to n.

[0066]

[0067] When t is equal to 1, is the default value in the Cross-BiLSTM enhanced model; when t is equal to n, Default value in the Cross-BiLSTM enhancement model.

[0068] That is, the first layer of forward hidden state representation related to the element at element position t is obtained by forward LSTM calculation The second layer backward hidden state representation is obtained by reverse LSTM calculation

[0069] Then, the first layer forward hidden state is represented as Concatenate with the second layer backward hidden state representation As the input of the Cross-BiLSTM enhanced model, the forward hidden state representation of the second layer is obtained through the forward LSTM calculation And the second layer backward hidden state representation is obtained by reverse LSTM calculation Then, the second layer forward hidden state is represented as And the backward hidden state representation By splicing, we can get the final hidden state H of the element at the current element position t t ;in, n represents the length of the input sequence, d h Represents the dimension of the hidden state.

[0070] Accordingly, according to the above iterative calculation results, the global sentence representation in, The second layer backward hidden state representation of the element at the first element position; is the second layer forward hidden state representation of the element at the nth element position, where n is the number of elements.

[0071] S116. Fuse the final hidden state of each element with the global sentence representation to obtain a global vector corresponding to each element.

[0072] Specifically, the global vector O corresponding to each element t =G||H t ;

[0073] Among them, O t is the global vector of the element at element position t; G is the global sentence representation; H t is the final hidden state representation of the element at element position t.

[0074] In order to fully extract the long-distance dependencies and global context information of the threat intelligence sequence, this paper first extracts the global sentence representation G from the output of Cross-BiLSTM, and the hidden state of each position step also contains the sequence context information, and fuses it with the global sentence representation into a global vector O t middle.

[0075] S118. Obtain different weights of the global vector according to the operation results of the global vector of each element and different weight matrices.

[0076] In the process of combining global information, the first weight matrix W is used H and the second weight matrix W G Assign different parameters, that is, first use the first weight matrix W H and the first bias term b H To calculate the linear mapping result of the local hidden state Then through the second weight matrix W G and the second bias term b G Get the linear mapping result of global information

[0077]

[0078] Among them, the first weight matrix W H , the second weight matrix W G , the first bias term b H , the second bias term b G It is the preset value after model training.

[0079] Different weights are calculated by sigmoid function and The final mapping range of the sigmoid function is (0,1), which is used to represent the probability distribution of weights.

[0080]

[0081] Among them, the different weights of the global vector include: the first weight related to the final hidden state and a second weight associated with the global sentence representation

[0082] S120, performing computational fusion on the final hidden state of each element and the global sentence representation according to different weights of the global vector to obtain a fused global feature representation.

[0083] Among them, the fused global feature representation Among them, ⊙ represents the element-wise product operation.

[0084] S122. Based on the fused global feature representation, obtain the label sequence probability; and output the entity type with the maximum probability value.

[0085] Predict the label sequence probability through the softmax function and predict the sequence Probability:

[0086]

[0087] Where W c ∈R n×k , W c represents the weight to be learned, n represents the maximum input sequence length (i.e., the length of the input sentence), and k represents the number of threat intelligence entity types; b c Represents the bias term, which can be a preset value after model training.

[0088] In order to obtain the optimal label sequence from the fused global feature representation, the entity type with the maximum probability value is selected through the argmax function. The calculation is as follows:

[0089]

[0090] Figure 3 It is a model schematic diagram of an entity recognition method according to an embodiment of the present disclosure.

[0091] In the model training process disclosed herein, cross entropy loss can be used to optimize the model and improve the prediction accuracy of the real entity type.

[0092]

[0093] Where N represents the number of training samples, k represents the number of threat intelligence entity types, and is a binary indicator with a value of 0 or 1, which is used to represent the input text w i Whether it belongs to the jth threat intelligence entity type.

[0094] The experimental environment of this disclosure is as follows: the processor model is Intel(R) Xeon(R) Platinum 8163CPU@2.50GHz, the GPU model is NVIDIA GeForce RTX 3090, the memory is 32GB, the Python version is 3.8.13, and the Pytorch version is 1.14. The SecureBERT pre-trained model used in this disclosure contains 12 hidden layers and 12 attention heads, and the hidden state size of Cross-BiLSTM is 500. In addition, Adam is used as the model optimizer, the Dropout parameter is 0.1, and the initial learning rate is set to 1e-5. The specific experimental parameter settings are shown in Table 1.

[0095] Table 1 Experimental parameter settings

[0096] Experimental parameters Parameter Value hidden size 500 Batch size 16 Epoch 20 Dropout rate 0.1 Optimizer method Adam Fine tuning True Learning rate 1e-5 Maximum length of input text 256

[0097] This paper uses two public threat intelligence datasets for experiments. First, DNRTI is a large-scale threat intelligence dataset for named entity recognition. It analyzes more than 300 threat reports with 182,452 words, consisting of 6,574 annotated sentences and 36,412 entities. The DNRTI dataset contains 13 label categories, namely hacker organization (HackOrg), attack (OffAct), sample public file (SamFile), security team (SecTeam), tool (Tool), time (Time), purpose (Purp), area (Area), industry (Idus), organization (Org), way (Way), vulnerability (Exp), and features (Features); the other dataset is the MalwareTextDB dataset, which is the first database of annotated malware. It is annotated by 39 APT reports, with a total of 6,819 sentences and 10,983 entities. The MalwareTextDB dataset contains 3 label categories, namely attack behavior (Action), entity (Entity) and modifier (Modifier). The statistical information of the dataset used in the experiment is shown in Table 2.

[0098] Table 2 Statistical results of experimental data sets

[0099]

[0100]

[0101] The experiment uses precision (P), recall (R) and F1-measure as evaluation indicators, where F1-measure is the harmonic mean of precision P and recall R. The calculation method of the experimental evaluation indicators is as follows:

[0102]

[0103] In the formula, TP represents the number of correctly identified threat intelligence entities, FP represents the number of incorrectly identified threat intelligence entities, and FN represents the number of unidentified threat intelligence entities.

[0104] In order to verify the effect of the TISCG model proposed in this disclosure, a total of 5 entity recognition benchmark models were selected for comparative analysis of recognition effects. The comparison models in the experiment include: BiLSTM-CRF, IDCNN-CRF, CNN-BiLSTM-CRF, BERT-BiLSTM-CRF and the TISCG model proposed in this disclosure. The specific experimental results are shown in Tables 3 and 4.

[0105] Table 3 Experimental comparison results of DNRTI dataset

[0106] Model P R F1 BiLSTM-CRF 74.07 75.98 75.01 IDCNN-CRF 74.42 77.40 75.88 CNN-BiLSTM-CRF 82.99 78.10 80.48 BERT-BiLSTM-CRF 85.11 83.48 84.29 TISCG 86.16 86.86 86.51

[0107] Table 4 Experimental comparison results of MalwareTextDB dataset

[0108]

[0109]

[0110] Among them, from the test results of each model on the DNRTI dataset in Table 3, it can be seen that the F1 of the model proposed in this disclosure is better than the other five models. The F1 values ​​of the BiLSTM-CRF, IDCNN-CRF and CNN-BiLSTM-CRF models reached 75.01%, 75.88% and 80.48% respectively, and the results differed by 5.47 percentage points at most, indicating that the use of character-level features can improve the named entity recognition effect; the F1 value of the BERT-BiLSTM-CRF model has been significantly improved after using the pre-trained word embedding model, indicating that the initialization word vector of the BERT model is better than the random initialization of BiLSTM, and can provide rich context information; compared with the BERT-BiLSTM-CRF model, the TISCG model extracts long-distance dependencies and global context information in the text, and the F1 value is increased by 2.22 percentage points, which is significantly better than other models, proving that the model of this disclosure has a strong ability to predict threat intelligence entities in the DNRTI dataset.

[0111] As shown in Table 4, the public model fully extracts character-level, word-level and sentence-level features to enhance the semantic features of threat intelligence, effectively alleviating the problem of vector differences of threat intelligence entities. Compared with the BERT-BiLSTM-CRF model, the F1 value of the public model is increased by 1.48 percentage points, and the precision P and recall R values ​​are improved to a certain extent. Based on the experimental comparison results of the two public data sets, the recognition effect of the public model on the DNRTI data set is higher than that on the MalwareTextDB data set because the number of entity categories and the number of instances of each category in the MalwareTextDB data set are less than those in the DNRTI data set. In addition, the sparsity of the MalwareTextDB data set also limits the prediction ability of the TISCG model for entities.

[0112] By comparison, it is found that when the TISCG model performs threat intelligence named entity recognition tasks, the dynamic word vectors generated by the SecureBERT pre-trained model can represent the semantic features of threat intelligence entities in more dimensions; the Cross-BiLSTM model more comprehensively captures the long-distance dependencies of threat intelligence sequence texts and obtains cross-context hidden layer feature vector representations; and uses fusion features to deeply mine the global context information of threat intelligence. In summary, the recall rate and F1 value of the TISCG model in the two public data sets have been significantly improved, and it can better distinguish nouns of different entity categories.

[0113] First, we verify the influence of the model hidden state size on the recognition effect. The experimental results are as follows: Figure 4 As shown in the figure. The analysis shows that as the hidden state size increases, the model recognition effect first increases and then decreases. When the hidden size parameter of Cross-BiLSTM is set to 500, the F1 value of the TISCG model on the DNRTI dataset reaches a peak, and then the model effect gradually decreases as the hidden state size increases. Therefore, the model disclosed in this paper sets the hidden state size to 500 to fully extract the dependencies between threat entities and provide better parameter fitting.

[0114] To verify the effectiveness of the TISCG model, the present disclosure conducted multiple ablation experiments on the DNRTI dataset. The baseline models used for comparison include: (1) "-SecureBERT", which removes the SecureBERT module based on the TISCG model and does not consider the model's language representation and semantic relationship understanding capabilities for threat intelligence text; (2) "-Cross", which removes the Cross-BiLSTM modeling capability for cross-context information based on the TISCG model and only considers the local hidden state of the long short-term memory network; (3) "-GCM", which uses SecureBERT and Cross-BiLSTM for feature extraction without adding weight coefficients. The experimental results are shown in Table 5.

[0115] Table 5 Ablation experiment results of DNRTI dataset

[0116] Model P R F1 -SecureBERT 76.64 84.30 80.29 -GCM 85.71 82.57 84.11 -Cross 85.03 86.81 85.91 TISCG 86.16 86.86 86.51

[0117] From the experimental results in the table, we can see that removing any of the modules, SecureBERT, Cross-BiLSTM, and weights, will reduce the recognition effect of the model. After removing the SecureBERT module, the model's ability to understand the semantics of threat intelligence has decreased, and the difference in the entity's feature vectors has caused the model's F1 to drop by 6.22 percentage points, indicating the effectiveness of the model's ability to extract knowledge features in the threat intelligence field; after removing the Cross-BiLSTM module, the features extracted for special context dependencies are insufficient, and the model cannot understand the distribution of global sentence features before fusion, resulting in a 0.6 percentage point drop in the F1 value, which verifies the effectiveness of Cross-BiLSTM in extracting long-distance dependencies of text and hidden layer features containing context information to improve the recognition effect of threat intelligence named entities; after removing the weights, the global feature representation is insufficient, and the model's F1 value drops by 2.4 percentage points, indicating that weights and fusion can effectively combine global context information and hidden state information features.

[0118] The entity recognition method disclosed in this paper obtains the dynamic word vector of the threat intelligence sequence through the SecureBERT pre-trained language model to reduce the feature vector difference of the entity; in order to fully capture the cross-context information of the sequence, a cross-bidirectional long short-term memory network is used to capture the long-distance dependency of the sequence; finally, the global context features with rich representation are obtained by combining fusion and weight calculation, and a suitable loss function is designed to ensure the effectiveness of model training. The experimental results show that compared with other methods, the method proposed in this paper has achieved better results on two threat intelligence public datasets, the DNRTI dataset and the MalwareTextDB dataset, verifying the effectiveness of the TISCG model proposed in this paper.

[0119] Any process or method description in the flowchart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the present disclosure. The processor performs the various methods and processes described above. For example, the method implementation in the present disclosure can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0120] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or in combination with such instruction execution systems, apparatuses, or devices.

[0121] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program may be printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a memory.

[0122] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0123] A person skilled in the art may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.

[0124] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.

[0125] The present disclosure also provides an electronic device, including: a memory, the memory storing execution instructions; and a processor or other hardware module, the processor or other hardware module executes the execution instructions stored in the memory, so that the processor or other hardware module executes the above method.

[0126] The present disclosure also provides a readable storage medium, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the above method.

[0127] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.< / mask>

Claims

1. An entity recognition method, characterized in that: include: Obtaining a threat intelligence sequence including a threat intelligence entity; Processing the threat intelligence sequence using the SecureBERT pre-trained language model to obtain a semantic feature matrix corresponding to the threat intelligence sequence; wherein the semantic feature matrix includes a plurality of elements; The semantic feature matrix is ​​processed using a Cross-BiLSTM enhancement model to obtain a final hidden state of each element in each semantic feature matrix; and a global sentence representation is obtained; The final hidden state of each element is fused with the global sentence representation to obtain a global vector corresponding to each element; According to the operation results of the global vector of each element and different weight matrices, different weights of the global vector are obtained; The final hidden state of each element is fused with the global sentence representation according to the different weights of the global vector to obtain the fused global feature representation; Based on the fused global feature representation, the label sequence probability is obtained, and the entity type with the maximum probability value is output.

2. The entity recognition method according to claim 1, characterized in that: Obtaining the final hidden state of each element in each semantic feature matrix includes: According to the current element, the first layer of forward hidden state representation is obtained by forward LSTM calculation and the second layer of backward hidden state representation is obtained by reverse LSTM calculation; The forward hidden state representation of the first layer and the backward hidden state representation of the second layer are concatenated as the input of the Cross-BiLSTM enhanced model, and the forward hidden state representation of the second layer is obtained by forward LSTM calculation, and the backward hidden state representation of the second layer is obtained by backward LSTM calculation; The second layer forward hidden state representation and the backward hidden state representation are concatenated to obtain the final hidden state of the current element.

3. The entity recognition method according to claim 2, characterized in that: The global sentence representation in, The second layer backward hidden state representation of the element at the first element position; is the second layer forward hidden state representation of the element at the nth element position, where n is the number of elements.

4. The entity recognition method according to claim 3, characterized in that: The final hidden state of each element is fused with the global sentence representation to obtain the global vector corresponding to each element: O t =G||H t ||; Among them, O t is the global vector of the element at element position t; G is the global sentence representation; H t is the final hidden state representation of the element at element position t.

5. The entity recognition method according to claim 4, characterized in that: According to the operation results of the global vector of each element and different weight matrices, different weights of the global vector are obtained, including: Among them, W H is the first weight matrix; W G is the second weight matrix; b H is the first bias term; b G is the second bias term; sigmoid() represents the function; where the different weights of the global vector include: is the first weight associated with the final hidden state; is the second weight associated with the global sentence representation.

Citation Information

Cited By

  • Network threat intelligence analysis attack scene graph (ASG) generation method based on deep learning and natural language processing

    CN119966695A

  • A Method for Generating Attack Scenario Graphs (ASGs) for Network Threat Intelligence Analysis Based on Deep Learning and Natural Language Processing

    CN119966695B