Small sample named entity recognition method and system based on prompt learning

Abstract

The invention provides a small sample named entity recognition method and system based on prompt learning, and belongs to the field of named entity recognition, and the method comprises the steps: S1, collecting a network threat intelligence report, and carrying out the preprocessing to obtain a corpus; constructing a vocabulary for pre-training a mask language model based on a corpus to obtain a pre-training model CTIBERT; s2, constructing a CTINER data set based on a corpus; s3, task prompts composed of identity prompts, background information and position prompts are constructed; s4, based on SBERT and Jaccard similarity, selecting a sentence with the closest similarity to the sequence X to be recognized from the CTINER data set for demonstration; and S5, inputting the task prompt, the SBERT, the Jaccard sentence demonstration and the to-be-identified sequence X into the named entity identification network together, and outputting a predicted entity tag. According to the method, the performance of the small sample named entity recognition model is improved.

Description

Technical Field

[0001] This invention belongs to the fields of three-dimensional geological modeling, computational geometry, computer graphics, numerical simulation and computation, and specifically relates to a few-sample named entity recognition method and system based on cue learning. Background Technology

[0002] In the field of natural language processing, named entity recognition (NER) has received widespread attention as an important downstream research task. With the development of deep learning in recent years, training neural networks using large-scale supervised data has become almost a standardized process. However, acquiring large-scale supervised data is expensive, and its quality is difficult to control. Therefore, applying small-sample NER is a promising research direction. In the field of cybersecurity, cyber threat intelligence helps professionals conduct security analysis and decision-making, thereby enabling timely detection and response to cyberattacks. However, since cyber threat intelligence is mostly in unstructured form, it is difficult to share and utilize. Therefore, transforming unstructured cyber threat intelligence into a structured form is crucial. Named entity recognition technology can extract key information and provide a foundation for building knowledge graphs. However, annotating cyber threat intelligence data is a very time-consuming and labor-intensive task, requiring the assistance of domain experts. Therefore, applying NER to small-sample threat intelligence can solve these problems.

[0003] Currently, the main methods for few-sample named entity recognition include:

[0004] 1. Metric-based learning method: By learning a metric space, entities of the same category are closer together in the space, while entities of different categories are farther apart.

[0005] 2. Transfer learning-based methods: Improve model performance with limited labeled data by transferring knowledge from related tasks or related domains.

[0006] 3. Hint-based learning approach: By activating general knowledge in a pre-trained language model, a hint-based learning pipeline is developed by constructing verbalizers and templates.

[0007] 4. Pre-training and fine-tuning based method: Capture the language features of the input sequence by using a pre-trained language model, and then fine-tune it using a small number of samples.

[0008] Currently, there is no work specifically for few-sample named entity recognition in the field of network threat intelligence. Most existing few-sample named entity recognition methods are based on general domains. Directly applying general domain methods to the field of network threat intelligence will lead to some incompatibility issues due to the domain characteristics of intelligence data, which will affect the performance of few-sample entity recognition.

[0009] 1. Metric learning is highly sensitive to the distribution of samples. When there is a large difference in entity categories between the training set and the test set, its performance will be affected.

[0010] 2. For network threat intelligence data, there is no other suitable knowledge that can be used for transfer learning.

[0011] 3. The cue-based learning method does not apply a domain-specific language model, and the cue engineering does not take into account the relationship with the input sequence.

[0012] 4. Pre-training and fine-tuning methods may overfit with a small number of training samples. At the same time, using general domain pre-trained language models may not be compatible with network threat intelligence data and may fail to fully capture the specific contextual semantics of the network threat intelligence domain.

[0013] In summary, existing technologies have certain shortcomings and problems in the small-sample named entity recognition of network threat intelligence. Summary of the Invention

[0014] To address the aforementioned technical problems, this invention provides a few-sample named entity recognition method based on cue learning, comprising the following steps:

[0015] Step S1: Collect network threat intelligence reports, preprocess them to obtain a corpus; construct a vocabulary based on the corpus for pre-training of the masked language model to obtain the pre-trained model CTIBERT;

[0016] Step S2: Construct the CTIER dataset based on the corpus;

[0017] Step S3: Construct a task prompt consisting of identity hints, background information, and location hints;

[0018] Step S4: Based on SBERT and Jaccard similarity, select SBERT sentence examples and Jaccard sentence examples from the CTINR dataset that have the closest similarity to the sequence X to be identified;

[0019] Step S5: Input the task prompt, SBERT and Jaccard sentence examples and the sequence to be identified X into the named entity recognition network built on the CTIBERT model, and output the predicted entity label.

[0020] Beneficial effects:

[0021] 1. Employing a language model with domain characteristics: This invention collects and processes a large amount of intelligence data to form the CTIER dataset, which is used to train the language model. This avoids the bias problems caused by using a general domain language model and achieves integration from data to language model to prompt template, minimizing the impact on entity recognition.

[0022] 2. By concatenating task prompts and sentence demonstrations before the original input sequence, this invention enables the representation of the original input sequence to better fit the context, improves the accuracy of input in the hidden state, and enhances the performance of the few-sample named entity recognition model.

[0023] 3. This invention realizes named entity recognition of network threat intelligence. By using a small amount of labeled data, unstructured network threat intelligence can be structured, saving the cost of manual processing, contributing to the cause of network security, and reducing manpower and time costs.

[0024] 4. The method of this invention is applicable to various special fields. By training a domain language model, it improves the performance of entity recognition and has wide adaptability and universality. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a few-sample named entity recognition method based on cue learning according to the present invention;

[0026] Figure 2 This is a schematic diagram of the CTIBERT model pre-training process;

[0027] Figure 3 A diagram illustrating data annotation;

[0028] Figure 4 A diagram illustrating the calculation of SBERT similarity scores;

[0029] Figure 5 To illustrate the construction of a sentence;

[0030] Figure 6 This is a schematic diagram of the BiLSTM structure;

[0031] Figure 7 A schematic diagram of the structure of a named entity recognition model;

[0032] Figure 8 This is a comparison chart of ablation experiment results;

[0033] Figure 9 Scatter plots of Jaccard similarity and SBERT similarity;

[0034] Figure 10 This is a structural block diagram of a few-sample named entity recognition system based on cue learning according to the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0036] Example 1

[0037] like Figure 1 As shown in the figure, an embodiment of the present invention provides a few-sample named entity recognition method based on cue learning, which includes the following steps:

[0038] Step S1: Collect network threat intelligence reports, preprocess them to obtain a corpus; build a vocabulary based on the corpus for pre-training of the masked language model to obtain the pre-trained model CTIBERT;

[0039] Step S2: Construct the CTINR dataset based on the corpus;

[0040] Step S3: Construct a task prompt consisting of identity hints, background information, and location hints;

[0041] Step S4: Based on SBERT and Jaccard similarity, select SBERT sentence examples and Jaccard sentence examples from the CTINR dataset that have the closest similarity to the sequence X to be identified;

[0042] Step S5: Input the task prompts, SBERT and Jaccard sentence examples, and the sequence to be identified X into the named entity recognition network built on the CTIBERT model, and output the predicted entity labels.

[0043] In one embodiment, step S1 above—collecting network threat intelligence reports, preprocessing them to obtain a corpus; constructing a vocabulary based on the corpus for pre-training a masked language model to obtain the pre-trained model CTIBERT—specifically includes:

[0044] This invention first collects network threat intelligence reports from major cybersecurity companies and GitHub open-source repositories. To ensure that the trained language model contains richer semantic knowledge, dirty data is removed when preparing the training corpus, resulting in a corpus containing 450,156 sentences.

[0045] Then, a vocabulary containing all unique words is constructed using the aforementioned corpus. For the named entity recognition task, the Masked Language Model (MLM) is chosen as the pre-training task for this invention. The BERT model includes two pre-training tasks: Masked Language Model (MLM) and Next Sentence Prediction (NSP). The goal of the Masked Language Model is to randomly select a certain proportion of words in each input sequence and replace them with mask tags, and the model must reasonably predict these masked words. The input sequence for the Next Sentence Prediction task consists of two sentences, and the model's goal is to predict whether the second sentence immediately follows the first. Since the purpose of this invention is named entity recognition, pre-training is only performed on the MLM task.

[0046] Finally, the model size and parameters were configured according to the actual data and examples, and the model was trained using algorithms such as backpropagation and gradient descent. The training process was optimized by adjusting hyperparameters. The final pre-trained CTIBERT model had a batch size of 8, a training epoch of 5, and a training loss of approximately 1.648. The training performance of the CTIBERT model was evaluated based on its training loss and experimental results in actual downstream tasks. Once the model reached its expected performance, the CTIBERT model weights and configuration were saved for fine-tuning.

[0047] Figure 2 This is a schematic diagram of the CTIBERT model pre-training process.

[0048] In one embodiment, step S2 above, constructing the CTINR dataset based on the corpus, specifically includes:

[0049] Step S21: Process all network threat intelligence reports into TXT format, and only retain the main text content;

[0050] The raw threat intelligence text data collected and labeled in the embodiments of the present invention are mainly APT analysis reports, which are all processed into TXT format, and only the main text content is retained;

[0051] Step S22: Format the main text content, segment it into sentences and words, and then delete dirty data;

[0052] Step S23: After filtering the data processed in step S22 based on the pyramid model, determine the entity type;

[0053] This invention first references the pyramid model, removing three types of low-value, easily manipulated data from the pyramid: file hash values, IP addresses, and domain name information. This is because attackers frequently alter their metrics, such as the MD5 hash of malicious files and the IP addresses of botnets, to evade detection. Therefore, IOC is only effective for a very short period of two days. The remaining three layers of the pyramid contain the following feature data:

[0054] Network or host characteristics include: URL, sample file, and operating system (OS).

[0055] Attack tools include: legitimate tools and malware.

[0056] Tactics, Techniques, and Procedures (TTP) include: vulnerabilities, campaigns, and malicious emails.

[0057] In addition, this invention also references the STIX 2.1 (Structured Threat information eXpression, STIX) specification and the threat information expression model, adding the entity type of Attack Target.

[0058] Step S24: Label the entities manually;

[0059] Because the CTINR dataset is specifically designed for the field of network threat intelligence, it contains unique entity types and domain-specific vocabulary. Therefore, automated annotation tools cannot accurately identify related named entities. Thus, this embodiment of the invention uses manual annotation with the open-source Docano annotation system. Data annotation examples are shown below. Figure 3 As shown.

[0060] Step S25: Convert the annotated file to TXT format, according to the annotation method;

[0061] To facilitate subsequent supervision tasks, this invention converts the JSONL format files exported from Doccano into TXT format. Furthermore, it performs the conversion according to the BIO word delimiter annotation method, enabling the dataset to be used in subsequent named entity recognition tasks.

[0062] Step S26: Clean the data and delete statements that do not contain entities;

[0063] Network intelligence has its own unique characteristics; its sentences are generally long, but the entity density is low, resulting in a large number of statements without entities. This invention removes these statements, increasing the overall proportion of entities and solving the problem of uneven distribution of data labels. Finally, the training set, test set, and validation set are divided in a 7:2:1 ratio.

[0064] Step S27: Supplement entity types with too few entities to obtain the CTIER dataset.

[0065] In APT reports, due to the nature of report writing and other factors, the number of certain entity types may be too low. Therefore, this invention supplements the data for entity types (EML and URL) with insufficient numbers based on statistical data results.

[0066] To ensure the rationality and reliability of small sample data, when selecting small sample data in the CTINR dataset, it is necessary to ensure that all entity types are included. For the K-shot setting, ensure that K sentences are extracted for each entity type during sampling.

[0067] This invention addresses the following issues with current open-source datasets: 1) outdated data that does not include emerging entities; 2) small data size; 3) entity labels that do not conform to the STIX 2.1 specification; and 4) uneven data distribution. Therefore, this invention selects a portion of the data from the corpus processed in step S2 to construct the dataset CTINR. Detailed information on the entity types of this dataset is shown in Table 1, and a total of 13 entity types are defined, as shown in Table 2.

[0068] Table 1 CTINER dataset

[0069]

[0070] Table 2 Entity Types in the CTINR Dataset

[0071]

[0072] In one embodiment, step S3 above: constructing a task prompt consisting of identity prompts, background information, and location prompts, specifically includes:

[0073] Step S31: In order to guide the behavior of the model and indicate the position of the language model itself, construct identity prompts;

[0074] For example, the identity prompt could be: "I am an outstanding cybersecurity expert as well as an excellent linguist."

[0075] Step S32: To enrich the contextual semantic information, specific background information is added. For the named entity recognition task, background information is added.

[0076] For example, the background information is: "This is a named entity recognition task for the field of cyber security. The task is to label the entity types in the given sentence. [Sub-Prompt]", where [Sub-Prompt] is a sub-prompt corresponding to the entity category, used to explain the definition of the entity type, as shown in Table 3.

[0077] Table 3 Entity Types and Their Sub-Hints

[0078]

[0079] Step S33: Add a location hint to indicate the location of the sentence example.

[0080] For example, the location prompt might be: "Below are some demonstrations, and you should make the correct prediction."

[0081] In one embodiment, step S4 above, which involves selecting SBERT sentence examples and Jaccard sentence examples from the CTINR dataset that have the closest similarity to the sequence X to be identified, based on SBERT and Jaccard similarity, specifically includes:

[0082] Step S41: Convert the sentence to be identified into a sentence vector, and calculate the sentence with the highest similarity to SBERT in the CTINR dataset as the SBERT sentence example. The calculation formula is as follows:

[0083] ;

[0084] in, and These represent the embedding vectors of two sentences, and These represent the lengths of the vectors;

[0085] The SBERT similarity calculation method of this invention uses the all-roberta-large-v1 pre-trained model from the Sentence-Transformers library. The SBERT model is based on BERT, encoding sentence-level information into BERT. It trains the model using similar and dissimilar sentence pairs and compares the losses, thereby making similar sentences closer in the embedding space. To obtain sentence-level representations, the SBERT model uses a Siamese network structure, processing two input sentences by sharing weights, and then uses cosine similarity to measure their similarity. The SBERT model uses the Mean Squared Error (MSE) loss function to calculate... and Cosine similarity between ,like Figure 4 As shown, where and These represent two sentences respectively. The mean squared error loss function is calculated using the following formula:

[0086] ;

[0087] in, Represents the number of samples. It is the first The actual similarity score of each sample It is the similarity score predicted by the SBERT model.

[0088] Step S42: Calculate the sentence with the highest similarity to the sentence to be identified and the Jaccard sentence in the CTINR dataset as a Jaccard example sentence. The calculation formula is as follows:

[0089] ;

[0090] Here, A and B represent the word sets of the two sentences respectively; the Jaccard similarity value ranges from [0,1].

[0091] For low-resource settings, this invention proposes a similarity-based approach to enhance sentence output representation. Although constructing sentence similarity representations is more complex than constructing entity representations, sentence representations contain richer knowledge. This involves constructing sentence-based representations using similarity-based methods, including those based on SBERT similarity and those based on Jaccard similarity. Figure 5 As shown. This is the sequence to be identified. Find the example sentence with the highest SBERT similarity and the example sentence with the highest Jaccard similarity, and place the obtained example sentence into the sequence to be identified. The vicinity of [the target]. This approach not only improves the model's contextual understanding, helping it better understand entities and their contextual relationships in the input sequence, but also increases data diversity, helping the model learn more generalized feature representations and enhancing its adaptability in low-resource scenarios.

[0092] The sentence demonstration method of this invention includes both SBERT-based and Jaccard-based methods. The SBERT-based method captures deep semantic information of sentences and has a strong ability to understand complex sentence structures and semantics. The Jaccard-based method, on the other hand, measures the similarity of word sets between two sentences, requiring no complex training or computation process and is suitable for preliminary similarity judgment. Therefore, combining the two methods leverages their respective strengths. Examples of sentence demonstrations are shown in Tables 4 and 5.

[0093] Table 4 Example 1

[0094]

[0095] Table 5 Example 2

[0096]

[0097] In one embodiment, step S5 above—inputting the task prompt, SBERT and Jaccard sentence examples, and the sequence to be identified X together into a named entity recognition network built on the CTIBERT model, and outputting the predicted entity labels—specifically includes:

[0098] A named entity recognition network is constructed based on the CTIBERT pre-trained model, including: the CTIBERT pre-trained model as a distributed expression layer, Bi-LSTM as a context encoding layer, and CRF as a label decoding layer;

[0099] Input the sequence to be recognized X, SBERT sentence example S, Jaccard sentence example J, and task prompt T into the named entity recognition network;

[0100] First, the corresponding entity vectors are extracted using the CTIBERT pre-trained model;

[0101] Then, a Bi-LSTM is formed by combining the forward and backward LSTMs, and the output is the hidden state vector concatenated from the forward and backward LSTMs. , , ,

[0102] ;

[0103] This invention employs Bi-LSTM for context encoding. Bi-LSTM is a network structure formed by combining forward LSTM and backward LSTM, used to process forward and backward sequences respectively. The most important structure in LSTM is the gate, which controls the amount of information passing through. Each LSTM unit contains an input gate, a forget gate, and an output gate, as well as a cell state, such as... Figure 6 As shown.

[0104] Among them, the input gate Information used to determine which cell state to include; candidate cell states are... The formula is as follows:

[0105] ;

[0106] ;

[0107] ;

[0108] ;

[0109] in, These represent the weight matrix and the bias term, respectively. This represents the sigmoid activation function, used to control the degree to which the door opens and closes; its value range is (0, 1). This represents the hyperbolic tangent activation function, with a value range of (-1, 1).

[0110] Forgotten Gate The formula used to determine forgotten information in a cellular state is as follows:

[0111] ;

[0112] Output gate Used to determine which state is passed to the hidden state. The information is provided in the following formula:

[0113] ;

[0114] ;

[0115] Cell state It is a carrier of information, through the transmission of information. and The regulation of information retention and forgetting is achieved through the following formula:

[0116] ;

[0117] Finally, the output of Bi-LSTM Input to CRF layer, output predicted entity labels:

[0118] Given the feature function and its weights The conditional probability formula is as follows:

[0119] ;

[0120] in, The normalization factor is used to ensure that the sum of the probabilities of all possible labeled sequences is 1. All possible labeled sequences are represented by... The sequence length is represented by... express:

[0121] ;

[0122] CRF is actually a probabilistic graphical model used for statistical modeling, and is often used as the label decoding layer in named entity recognition models, such as... Figure 7 As shown in the named entity recognition model. For the sequence to be recognized Hidden state concatenation vector The corresponding output sequence can be obtained through CRF. The conditional probability distribution of is expressed as That is, given the observation sequence Labeled sequences under the condition The probability distribution;

[0123] Simultaneously, construct the cross-entropy loss function. This is used to measure the difference between the predicted label distribution and the true label distribution.

[0124] ;

[0125] in, represent The number of tokens in the data. represent The corresponding label sequence, Indicates that it is in a hidden state. Under these conditions, model hyperparameters generate The conditional probability.

[0126] To verify the recognition performance of this invention, under low-resource conditions, this invention performs random sampling from the training set of the CTIER dataset, sampling for each entity type. One sentence.

[0127] 1. The comparative model structure of the Named Entity Recognition Network (PROMPT-CTIBERT-Bi-LSTM-CRF) experiment of the present invention is CTIBERT-Bi-LSTM-CRF, that is, without any hints. The experimental results are shown in Table 6. This represents the number of sentences selected for each entity type.

[0128] Table 6. K-shot Experimental Results (Validation Set)

[0129]

[0130] As shown in Table 6, adding prompts to the model improves the F1 score. The smallest improvement is seen with 15-shot models, increasing the F1 score by 0.83%. The largest improvement is seen with 4-shot models, increasing the F1 score by 9.13%. The experimental results demonstrate that the PROMPT-CTIBERT-Bi-LSTM-CRF model is effective for... The improvement is more pronounced on datasets with smaller values.

[0131] 2. Ablation Experiment Results and Analysis

[0132] To verify the effectiveness of task prompts and sentence demonstrations in few-sample prompting learning methods, this invention conducted an ablation study, and the experimental results are shown in the figure below. Three types of experiments were conducted: TJSX (TXJS), JSX, and TX. This represents a task prompt. Representing Jaccard's demonstration, Representing SBERT as an example, This represents the original input sequence.

[0133] The TJSX (TXJS) experiment represents the PROMPT-CTIBERT-Bi-LSTM-CRF model proposed in this paper. The JSX experiment represents task prompt ablation, and the TX experiment represents sentence demonstration ablation.

[0134] Experimental results are as follows Figure 8 As shown, the experimental results of the TX model are generally at the lowest end, indicating that sentence demonstration plays a crucial role in the PROMPT-CTIBERT-Bi-LSTM-CRF model. Although the experimental results of JSX are sometimes good, overall, combining task prompts and sentence demonstration can maximize the performance of named entity recognition for few-shot models, especially for different... The value has good versatility.

[0135] The similarity scores of Jaccard and SBERT were also compared, such as... Figure 9As shown in the figure, it can be observed that there are very few cases where the Jaccard similarity and SBERT similarity are very close. On the contrary, there are certain differences in the similarity between the two among different samples, which indicates the complementarity between the two and verifies the rationality of the method in this paper.

[0136] Furthermore, the experiment revealed that the position of the sentence demonstration also affects model performance. The experimental results are shown in Table 7. In most cases, the TJSX model outperforms TXJS. This invention speculates that the reason is that the TJSX model places all prompts in the original input sequence. Previously, this was a prefix suggestion, and TJS would handle it. This causes an impact, while the TXJS model places the JS code within the original input sequence. The following is a suffix suggestion, for The impact will be reduced, thus affecting the model's performance. In addition, this paper speculates that the experimental results may be affected by the fact that SBERT similar sentences are sometimes empty, or by the influence of the language model's internal representation. Therefore, the PROMPT-CTIBERT-Bi-LSTM-CRF model selects the best experimental results from TJSX and TXJS. Experimental results on different sample sets show that placing the sentence example before the input sequence better represents the token, thereby improving entity recognition performance.

[0137] Table 7. Results of the comparative experiment at the demonstration locations (F1 score)

[0138]

[0139] Furthermore, comparative experiments were conducted on multiple publicly available datasets and the CTIBERT dataset constructed in this invention. The model structure was CTIBERT-BiLSTM-CRF, and the comparison model was BERT-BiLSTM-CRF, to verify the effectiveness of the CTIBERT model. The experimental results are shown in Table 8. By observing the entity recognition performance on the four datasets, it was found that the F1 score improved by 0.63%, 0.17%, 1.29%, and 1.49%, respectively, demonstrating the effectiveness of the method of this invention.

[0140] Table 8 Comparison Experiments on the CTIBERT Dataset

[0141]

[0142] Example 2

[0143] like Figure 10 As shown, this embodiment of the invention provides a few-shot named entity recognition system based on cue learning, comprising the following modules:

[0144] The pre-trained model module 61 is used to collect network threat intelligence reports, perform preprocessing, and obtain a corpus; based on the corpus, a vocabulary is constructed for the pre-training of the masked language model, resulting in the pre-trained model CTIBERT.

[0145] The dataset construction module 62 is used to construct the CTIER dataset based on the corpus.

[0146] A task prompting module 63 is constructed to create a task prompt consisting of identity prompts, background information, and location prompts.

[0147] A sentence demonstration module 64 is constructed to select SBERT sentence demonstrations and Jaccard sentence demonstrations that have the closest similarity to the sequence X to be identified from the CTINR dataset, based on SBERT and Jaccard similarity.

[0148] A named entity recognition network module 65 is constructed to input the task prompt, SBERT and Jaccard sentence examples and the sequence to be recognized X into the named entity recognition network constructed based on the CTIBERT model, and output the predicted entity labels.

[0149] A few-shot named entity recognition device based on cue learning includes one or more electronic devices, wherein the one or more electronic devices are used to implement the few-shot named entity recognition method based on cue learning.

[0150] An electronic device includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a few-sample named entity recognition method based on cue learning.

[0151] A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to implement a few-sample named entity recognition method based on cue learning.

[0152] A non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a few-sample named entity recognition method based on cue learning.

[0153] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A few-sample named entity recognition method based on cue learning, characterized in that, include: Step S1: Collect network threat intelligence reports, preprocess them, and obtain a corpus; A vocabulary was constructed based on the corpus for pre-training of the masked language model, resulting in the pre-trained model CTIBERT. Step S2: Construct the CTIER dataset based on the corpus; Step S3: Construct a task prompt consisting of identity hints, background information, and location hints; Step S4: Based on SBERT and Jaccard similarity, select SBERT sentence examples and Jaccard sentence examples from the CTINR dataset that have the closest similarity to the sequence X to be identified; Step S5: Input the task prompt, SBERT and Jaccard sentence examples and the sequence to be identified X into the named entity recognition network built on the CTIBERT model, and output the predicted entity label.

2. The few-sample named entity recognition method based on cue learning according to claim 1, characterized in that, Step S2: Constructing the CTINR dataset based on the corpus, specifically including: Step S21: Process all the network threat intelligence reports into TXT format, and only retain the main text content; Step S22: Format the main text content, segment it into sentences and words, and then delete dirty data; Step S23: After filtering the data processed in step S22 based on the pyramid model, determine the entity type; Step S24: Label the entities manually; Step S25: Convert the annotated file to TXT format, according to the BIO term delimitation method; Step S26: Clean the data and delete statements that do not contain entities; Step S27: Supplement entity types with too few entities to obtain the CTIER dataset.

3. The few-sample named entity recognition method based on cue learning according to claim 2, characterized in that, Step S3: Constructing a task prompt consisting of identity prompts, background information, and location prompts, specifically including: Step S31: In order to guide the behavior of the model and indicate the position of the language model itself, construct identity prompts; Step S32: To enrich the contextual semantic information, specific background information is added. For the named entity recognition task, background information is added. Step S33: Add a location hint to indicate the location of the sentence example.

4. The few-sample named entity recognition method based on cue learning according to claim 3, characterized in that, Step S4: Based on SBERT and Jaccard similarity, select SBERT sentence examples and Jaccard sentence examples with the closest similarity to the sequence X to be identified from the CTINR dataset, specifically including: Step S41: Convert the sentence to be identified into a sentence vector, and calculate the sentence with the highest similarity to SBERT in the CTINR dataset as the SBERT sentence example. The calculation formula is as follows: ； in, and These represent the embedding vectors of two sentences, and These represent the lengths of the vectors; Step S42: Calculate the sentence with the highest similarity to the sentence to be identified and the Jaccard sentence in the CTINR dataset as a Jaccard sentence example. The calculation formula is as follows: ； Here, A and B represent the sets of words in the two sentences, respectively.

5. The few-sample named entity recognition method based on cue learning according to claim 4, characterized in that, Step S5: Input the task prompt, SBERT and Jaccard sentence examples, and the sequence to be identified X into a named entity recognition network built on the CTIBERT model, and output the predicted entity labels, specifically including: A named entity recognition network is constructed based on the CTIBERT pre-trained model, including: the CTIBERT pre-trained model as a distributed expression layer, Bi-LSTM as a context encoding layer, and CRF as a label decoding layer; The sequence to be identified X, SBERT sentence example S, Jaccard sentence example J, and task prompt T are input into the named entity recognition network. First, the corresponding entity vectors are extracted using the CTIBERT pre-trained model; Then, a Bi-LSTM is formed by combining the forward and backward LSTMs for encoding, outputting the hidden state vector after concatenating the forward and backward LSTMs. , , , Concatenate these four vectors together: ; Finally, Input the CRF layer and output the predicted entity labels; Constructing the cross-entropy loss function This is used to measure the difference between the predicted label distribution and the true label distribution. ； in, represent The number of tokens in the data. represent The corresponding label sequence, Indicates that it is in a hidden state. Under these conditions, model hyperparameters generate The conditional probability.

6. A few-sample named entity recognition system based on cue-based learning, characterized in that, Includes the following modules: The pre-trained model module is used to collect network threat intelligence reports, preprocess them, and obtain a corpus; a vocabulary is constructed based on the corpus for the pre-training of the masked language model, resulting in the pre-trained model CTIBERT. A dataset construction module is used to construct the CTINR dataset based on the corpus; A task prompting module is built to construct task prompts consisting of identity prompts, background information, and location prompts; A sentence demonstration module is constructed to select SBERT sentence demonstrations and Jaccard sentence demonstrations that have the closest similarity to the sequence X to be identified from the CTINR dataset, based on SBERT and Jaccard similarity. A named entity recognition network module is constructed to input the task prompt, SBERT and Jaccard sentence examples, and the sequence to be recognized X into the named entity recognition network built based on the CTIBERT model, and output the predicted entity labels.

7. A few-sample named entity recognition device based on cue-based learning, characterized in that, It includes one or more electronic devices, wherein the one or more electronic devices are used to implement the method of any one of claims 1 to 5.

8. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 5.

Images