A knowledge extraction method for network security intent understanding, electronic equipment and storage medium

Abstract

A knowledge extraction method for network security intention understanding, an electronic device and a storage medium belong to the technical field of network security. In order to solve the problem of accurate extraction of knowledge in artificial intelligence network security threat analysis report. The present application comprises collecting network security threat analysis report original text, carrying out data preprocessing; selecting a pre-training model, inputting the processed word sequence into the selected pre-training model for mask language model (MLM) task training; based on the selected pre-training model, the model is adjusted, the entity extraction sub-model and the relation extraction sub-model are added downstream of the pre-training model, and the trained knowledge extraction model for network security threat analysis report is obtained by using the obtained training result for model training; the trained knowledge extraction model for network security threat analysis report is used for model prediction, result verification and optimization. The present application improves the applicability of the model in the field of network security.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence network security, and particularly relates to a knowledge extraction method for network security intention understanding, an electronic device and a storage medium. BACKGROUND

[0002] The network space security situation in the world today is complex and changing rapidly, and politics, economy, military, culture and other aspects are affected by network space security at all times. In recent years, network security incidents have occurred frequently, and increasingly large-scale and dynamic attack operations have brought great challenges to national network space security protection work.

[0003] At the same time, with the rapid development of artificial intelligence technology, NLP technologies such as knowledge extraction have provided new ideas for network security practitioners. There are a large amount of network security related data and text in the Internet, such as Internet security forums and security threat analysis reports published by manufacturers. However, the knowledge existing in the text depends on manual processing and cannot be directly understood and utilized by computers. Through knowledge extraction technology, relevant information can be automatically discovered and extracted from multiple heterogeneous texts, so that computers can understand and process it more intelligently, and then the scattered and distributed unstructured network security data can be integrated into structured network security intelligence knowledge, so as to effectively manage, understand and organize massive intelligence information.

[0004] At present, the existing knowledge extraction technology in the market mostly focuses on text in general fields, and the pre-training-fine-tuning paradigm has become a widely used mainstream methodology. The pre-training model is first pre-trained on a large corpus dataset in the general field, and through a large amount of pre-training, it has the basic ability to extract data features. In the fine-tuning stage, it does not need too much labeled corpus to achieve good results, and can save a lot of training time. However, there are some differences between the general field and the network security field corpus, such as professional terms and hash values and other irregular characters, in addition, the corpus of the general field only contains one or two sentences, and the pre-training stage ignores the possible chapter structure in the network security report. This may lead to poor results when directly using the model to predict network security threat analysis reports. Therefore, it is particularly important to develop a knowledge extraction method for network security intention understanding. SUMMARY

[0005] The problem to be solved by the present application is to accurately extract knowledge from network security threat analysis reports, and a knowledge extraction method for network security intention understanding, an electronic device and a storage medium are proposed.

[0006] To achieve the above purpose, the technical scheme is as follows:

[0007] A knowledge extraction method for network security intent understanding includes the following steps:

[0008] S1. Collect the raw text of the cybersecurity threat analysis report, which is divided into training data and prediction data;

[0009] S2. Perform data preprocessing on the training data of the original text of the collected network security analysis report, including IOC lexical replacement and structural lexical insertion. Then, use a word segmenter to divide the processed original text to obtain the processed lexical sequence as the training set.

[0010] S3. Select a pre-trained model, input the processed word sequence obtained in step S2 into the selected pre-trained model to train the masked language model (MLM) task, and obtain the model weights after further pre-training.

[0011] S4. Based on the pre-trained model selected in step S3, the model is adjusted. An entity extraction sub-model and a relation extraction sub-model are added downstream of the pre-trained model. The model weights obtained in step S3 are used to train the model, resulting in a trained knowledge extraction model for network security threat analysis reports.

[0012] S5. Using the prediction data of the original text of the cybersecurity threat analysis report obtained in step S1, perform model prediction on the trained knowledge extraction model for the cybersecurity threat analysis report, and verify and optimize the results.

[0013] Furthermore, the specific implementation method of step S2 includes the following steps:

[0014] S2.1. Perform IOC token replacement on the training data of the original text of the collected network security analysis report. Replace the hash value, URL, domain name, and IP address mentioned in the text with the corresponding tokens [HASH], [URL], [DOMAIN], and [IP] using regular expression replacement.

[0015] S2.2. Then, insert structural tokens. Insert a token representing the structure of the line in the text at the beginning of each line. Set [h1], [h2], [p], and [tr] to represent first-level headings, second-level headings, body paragraphs, and table rows, respectively.

[0016] S2.3. Use a word segmenter to divide the processed text into a sequence of tokens T = {w1, w2, w3, ..., w...} n}, where w n For the nth word, the sequence length does not exceed the maximum length of the pre-trained model.

[0017] Further, the network security analysis report original text exceeding the maximum length of the pre-trained model is first divided into multiple segments meeting the length requirement in step S2.1.

[0018] Further, the specific implementation method of step S3 includes the following steps:

[0019] S3.1. Select BERT as the pre-trained model, and change the word segmenter of the pre-trained model for dividing the token sequence;

[0020] S3.2. Set the input text as the processed token sequence T obtained in step S2, perform a mask operation, define a mask function M(T, p) that accepts the processed token sequence T and a mask probability p as input, and the mask function replaces the token in the text with a verification code mark [MASK] with a mask probability p. The masked sequence T' is represented as:

[0021] T' = {w'1, w'2, w'3,..., w' n}

[0022] where w' n is the nth masked token;

[0023] S3.3. Use the pre-trained model selected in step S3.1 to accept the masked sequence as input, perform a mask language model MLM task training, and output the prediction distribution of the masked word, expressed as:

[0024] P(w i | T') = f(T')[i]

[0025] where f(T')[i] represents the prediction distribution output by the pre-trained model at position i, and P(w i | T') is the prediction distribution of the ith masked token when the input is T'.

[0026] For each masked token, calculate the cross-entropy loss between the prediction distribution of the pre-trained model and the real word distribution, expressed as:

[0027] L i = -log P(w i | T')

[0028] where L i is the cross-entropy loss at position i.

[0029] Then sum the losses of all masked positions to obtain the loss function L, expressed as:

[0030] L = ∑L i ;

[0031] Then, according to the loss function L, the parameters of the pre-trained model are updated by a back propagation algorithm to obtain the model weight after further pre-training.

[0032] Further, the specific implementation method of step S4 includes the following steps.

[0033] S4.1. Adding an entity extraction sub-model downstream of the pre-trained model, using a conditional random field (CRF) as the entity extraction sub-model, and the expression is as follows:

[0034]

[0035] wherein x is an input sequence, P(y|x) is a conditional probability distribution of an output sequence y given an input sequence x; f k (y i-1 ,y i ,x,i) is a feature function, λ k is a weight of the feature function; and Z(x) is a normalization factor.

[0036] The input sequence x=BERT(T) is set as the output feature vector sequence of the pre-trained model, y is a BIO label corresponding to each word element, and the normalization factor is obtained by summing all possible output sequences, and the expression is as follows:

[0037]

[0038] wherein y' represents all possible values of the output label sequence, y' i i is a label of the i-th word element.

[0039] S4.2. Using a feedforward neural network (FFNN) as a relation extraction sub-model, obtaining the positions of each entity according to the output of the entity extraction sub-model, averaging the feature vectors of all word elements of each entity x ei , and combining and splicing the feature vectors corresponding to the word elements x [CLS] at the beginning of each entity to obtain the input x of the relation extraction sub-model, and the expression is as follows:

[0040] x=x [CLS] ||x ei ||x ej ;

[0041] Then, the feedforward neural network outputs the relationship type distribution between the entity pairs, and the expression is as follows:

[0042] F(x)=max(0,xW1+b1)W2+b2

[0043] Wherein, W1, W2 are respectively a weight matrix of a first hidden layer, a weight matrix of a second hidden layer of the feedforward neural network model, b1, b2 are respectively a bias term of the first hidden layer, a bias term of the second hidden layer of the feedforward neural network model;

[0044] S4.3. Training the entity extraction sub-model using the labeled data set, setting a CRF loss function L CRF , the expression is:

[0045] L CRF =-logP(y|x);

[0046] Setting a FFNN loss function L FFNN , the expression is:

[0047] L FFNN =-∑ i y i ×log(y' i );

[0048] Wherein, y i is the real relationship type, y' i is the relationship type predicted by FFNN;

[0049] Finally, based on the CRF loss function L CRF And the FFNN loss function L FFNN Calculate the weighted average L', the expression is:

[0050] L'=λ'L CRF +(1-λ')L FFNN

[0051] Wherein, λ' is the weight value corresponding to the weighted average.

[0052] An electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the knowledge extraction method for network security intent understanding when executing the computer program.

[0053] A computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the knowledge extraction method for network security intent understanding.

[0054] The beneficial effects of the present application are:

[0055] The knowledge extraction method for network security intention understanding provided by the application is based on an advanced pre-training model, and through continuing pre-training on network security field corpus and fine-tuning to adapt to the specific needs of network security threat analysis reports, efficient and accurate knowledge extraction is realized from the network security threat analysis reports. This method not only utilizes the powerful language understanding and generation capability of the pre-training model, but also improves the applicability of the model in the network security field through targeted continuing pre-training and fine-tuning.

[0056] The knowledge extraction method for network security intention understanding provided by the application eliminates the gap between general corpus and network security corpus by pre-processing data and adding a continuing pre-training link, that is, the model first continues the pre-training task on the pre-processed network security vertical field unlabeled corpus, and then fine-tunes for the downstream task to obtain the final prediction model. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The flowchart of the knowledge extraction method for network security intention understanding provided by the application;

[0058] Figure 2 The structural block diagram of the knowledge extraction method for network security intention understanding provided by the application;

[0059] Figure 3 The knowledge extraction model structure schematic diagram for network security threat analysis reports provided by the application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical scheme and advantages of the application clearer, the application is further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and are not used to limit the application, that is, the described specific embodiments are only a part of the embodiments of the application, but not all the specific embodiments. The components of the specific embodiments of the application described and shown in the drawings can be arranged and designed in various different configurations, and the application can also have other embodiments.

[0061] Therefore, the detailed description of the specific embodiments of the application provided in the drawings below is not intended to limit the scope of the claimed application, but only represents selected specific embodiments of the application. Based on the specific embodiments of the application, all other specific embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0062] In order to further understand the inventive content, characteristics and effects of the application, the following specific embodiments are exemplified, and the attached Figure 1 -ATTACHMENTFigure 3 The detailed description is as follows:

[0063] Embodiment 1:

[0064] A knowledge extraction method for network security intent understanding, comprising the following steps:

[0065] S1. Collect network security threat analysis report original text, divided into training data and prediction data;

[0066] S2. Data preprocessing is performed on the training data of the collected network security analysis report original text, including IOC token replacement, structure token insertion, and then using a tokenizer to divide the processed original text to obtain a processed token sequence as a training set;

[0067] Further, the specific implementation method of step S2 comprises the following steps:

[0068] S2.1. Replace the training data of the collected network security analysis report original text with IOC tokens, replace the hash value, URL, domain name, IP address mentioned in the text with the corresponding tokens [HASH], [URL], [DOMAIN], [IP] through regular replacement;

[0069] Further, in step S2.1, first divide the network security analysis report original text exceeding the maximum length of the pre-trained model into multiple segments meeting the length requirement;

[0070] S2.2. Then insert a structure token at the beginning of each line of the text, set [h1], [h2], [p], [tr] to represent the structure of the line in the text, respectively representing the first-level title, the second-level title, the text paragraph and the table row;

[0071] S2.3. Use a tokenizer to divide the processed text into a token sequence T={w1, w2, w3, …, w n}, wherein w n is the nth token, and the sequence length does not exceed the maximum length of the pre-trained model;

[0072] S3. Select a pre-trained model, input the processed token sequence obtained in step S2 into the selected pre-trained model for mask language model (MLM) task training to obtain the model weight after continuous pre-training;

[0073] Further, the specific implementation method of step S3 comprises the following steps:

[0074] S3.1. Select BERT as the pre-trained model, change the tokenizer of the pre-trained model for dividing the token sequence;

[0075] Further, the selected pre-training model is BERT, which is a bidirectional pre-training language representation model based on Transformer. For each input token sequence T, the output BERT(T) has a one-to-one correspondence with the input sequence, that is, each output vector represents the representation information of the corresponding token;

[0076] S3.2. Set the input text as the processed token sequence T obtained in step S2, perform a mask operation, define a mask function as M(T, p), accept the processed token sequence T and the mask probability p as inputs, and the mask function replaces the token in the text with a verification code mark [MASK] with a mask probability p. The masked sequence T' is represented as:

[0077] T' = {w'1, w'2, w'3, …, w' n}

[0078] where w' n is the n-th masked token;

[0079] S3.3. Use the pre-training model selected in step S3.1 to accept the masked sequence as input and perform a masked language model MLM task training, output the prediction distribution of the masked word, expressed as:

[0080] P(w i ∣T') = f(T')[i]

[0081] where f(T')[i] represents the prediction distribution output by the pre-training model at position i, and P(w i ∣T') is the prediction distribution of the i-th masked token when the input is T'.

[0082] For each masked token, calculate the cross-entropy loss between the prediction distribution of the pre-training model and the real word distribution, expressed as:

[0083] L i = -log P(w i ∣T')

[0084] where L i is the cross-entropy loss at position i.

[0085] Then sum the losses of all masked positions to obtain the loss function L, expressed as:

[0086] L = ∑L i ;

[0087] Then update the parameters of the pre-training model according to the loss function L through the back propagation algorithm to obtain the model weight after further pre-training.

[0088] S4. Model adjustment based on the pre-trained model selected in step S3, adding entity extraction sub-model and relation extraction sub-model downstream of the pre-trained model, using the model weight obtained after continuing pre-training in step S3 to perform model training, and obtaining a trained knowledge extraction model for network security threat analysis report;

[0089] Further, task definition: define the types of specific entities and relationships that need to be recognized in the knowledge extraction task, such as "software", "intruder", "attack means", "defense means", "identity", "intrusion indicator", and "use", "target", "impersonate", "possible correlation", etc.

[0090] Further, data annotation: annotate the positions of the entities defined in the task and the types of relationships between the entities on the training corpus, and preprocess them for model training.

[0091] Further, downstream model construction: add new model layers downstream of the pre-trained model for decoding the hidden state of the pre-trained model and outputting the recognized entity relationship triplets.

[0092] Further, the specific implementation method of step S4 includes the following steps:

[0093] S4.1. Add an entity extraction sub-model downstream of the pre-trained model, use Conditional Random Field (CRF) as the entity extraction sub-model, and the expression is:

[0094]

[0095] where x is the input sequence, P(y|x) is the conditional probability distribution of the output sequence y given the input sequence x; f k (y i-1 ,y i ,x,i) is a feature function, λ k is the weight of the feature function; Z(x) is the normalization factor;

[0096] Set the input sequence x = BERT(T) as the output feature vector sequence of the pre-trained model, y as the BIO label corresponding to each word element, and the normalization factor is obtained by summing all possible output sequences, and the expression is:

[0097]

[0098] where y' represents all possible values of the output label sequence, y' i i is the label of the i-th word element.

[0099] S4.2. Using a feed-forward neural network (FFNN) as the relation extraction sub-model, obtaining the position of each entity according to the output of the entity extraction sub-model, averaging the feature vectors of all word pieces of each entity x ei , and combining them two by two with the first word piece x [CLS] The corresponding feature vectors are spliced to obtain the input x of the relation extraction sub-model, and the expression is:

[0100] x = x [CLS] || x ei || x ej ;

[0101] Then the relation type distribution between the entity pairs is output by the feed-forward neural network, and the expression is:

[0102] F(x) = max(0, xW1 + b1)W2 + b2

[0103] Where W1 and W2 are the weight matrix of the first hidden layer and the weight matrix of the second hidden layer of the feed-forward neural network model, respectively, and b1 and b2 are the bias term of the first hidden layer and the bias term of the second hidden layer of the feed-forward neural network model, respectively.

[0104] S4.3. The entity extraction sub-model is trained using the labeled data set, and the CRF loss function L CRF is set, and the expression is:

[0105] L CRF = -logP(y|x);

[0106] The FFNN loss function L FFNN is set, and the expression is:

[0107] L FFNN = -∑ i y i ×log(y' i );

[0108] Where y i is the true relation type, and y' i is the relation type predicted by FFNN;

[0109] Finally, the weighted average L' is calculated based on the CRF loss function L CRF and the FFNN loss function L FFNN , and the expression is:

[0110] L' = λ'L CRF + (1-λ')L FFNN

[0111] Where λ' is the weight value corresponding to the weighted average.

[0112] Further, back propagation: update the parameters of the model according to the weighted average L' through the back propagation algorithm.

[0113] S5. Use the predicted data of the network security threat analysis report original text obtained in step S1 to perform model prediction on the trained knowledge extraction model for network security threat analysis reports, and perform result verification and optimization.

[0114] Further, input processing: perform the aforementioned preprocessing work on the network security threat analysis report that needs to be subjected to knowledge extraction as input text. Model inference: use the fine-tuned model to infer the input text, identify and extract key information, and the entity extraction sub-model outputs the BIO label of the word token sequence, and the relationship extraction sub-model outputs the relationship type distribution between entity pairs. Post-processing: post-process the extracted information to obtain the type of entity according to the BIO label, and then obtain the entity relationship triple according to the relationship type. After deduplication, merging, formatting, etc., a structured knowledge representation is generated.

[0115] Further, result verification and optimization includes:

[0116] Manual review: manually review the extraction results to ensure the accuracy and completeness of the information;

[0117] Feedback loop: based on the manual review results, further fine-tune the model to improve the accuracy of knowledge extraction.

[0118] Continuous update: as the network security threats continue to evolve, regularly update the labeled data set and the model to maintain the timeliness and accuracy of the method.

[0119] Embodiment 2:

[0120] An electronic device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps of the knowledge extraction method for network security intent understanding according to the embodiment 1.

[0121] The computer device of the present application can be a device including a processor and a memory, such as a single-chip microcomputer including a central processing unit. And the processor is used to execute the computer program stored in the memory to realize the steps of the knowledge extraction method for network security intent understanding.

[0122] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0123] The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0124] Embodiment 3:

[0125] A computer readable storage medium, having stored thereon a computer program, the computer program being executed by a processor to implement the knowledge extraction method for network security intent understanding according to the embodiment 1.

[0126] The computer readable storage medium of the present application can be any form of storage medium readable by the processor of the computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc., and the computer readable storage medium has stored thereon a computer program, when the processor of the computer device reads and executes the computer program stored in the memory, the steps of the knowledge extraction method for network security intent understanding can be implemented.

[0127] The computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB (Universal Serial Bus) flash disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a ROM (Read-Only Memory), a RAM (Random Access Memory), an electrical carrier signal, a telecommunications signal, and a software distribution medium, and the like. It should be noted that the computer readable medium contains contents that can be appropriately added, deleted or modified according to requirements of legislation and patent practices in a jurisdiction, for example, in some jurisdictions, according to legislation and patent practices, the computer readable medium does not include an electrical carrier signal and a telecommunications signal.

[0128] It should be noted that the relational terms herein, such as first and second, and the like, are used solely to distinguish one from another entity or action, without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0129] Although the present application has been described with reference to specific implementations, it is apparent that various modifications and changes can be made thereto without departing from the scope of the present application. In particular, various features of the specific implementations of the present application can be interchanged and otherwise used in other specific implementations, without departing from the scope of the present application. Such modifications and changes are intended to fall within the scope of the application. Therefore, the present application is not limited to the specific implementations described herein, but only by the scope of the appended claims, and their equivalents.

Claims

1. A knowledge extraction method for network security intent understanding, characterized in that, Comprising the following steps: S1. Collect network security threat analysis report original text, divided into training data and prediction data; S2. Data preprocessing is performed on the training data of the collected network security analysis report original text, including IOC token replacement, structure token insertion, and then using a tokenizer to divide the processed original text to obtain a processed token sequence as a training set; S3. Select a pre-training model, input the processed token sequence obtained in step S2 into the selected pre-training model for mask language model MLM task training to obtain the model weight after further pre-training; The specific implementation method of step S3 comprises the following steps: S3.

1. Select BERT as the pre-training model, and change the tokenizer of the pre-training model for dividing the token sequence; S3.

2. Set the input text as the processed token sequence T obtained in step S2, perform a masking operation, define a masking function M(T, p) that accepts the processed token sequence T and a masking probability p as input, and the masking function replaces the token in the text with a verification code mark [MASK] with a masking probability p. The masked sequence T' is represented as: T' = {w'1, w'2, w'3,..., w'N}; and n} where w n is the nth masked wordpiece; S3.

3. Use the pre-training model selected in step S3.1 to accept the masked sequence as input and perform mask language model MLM task training, output the prediction distribution of the masked word, expressed as: P(w i ∣T’)=f(T’)[i] where f(T') [i] represents the prediction distribution output by the pre-trained model at position i, P(w i | T') is the prediction distribution for the i-th masked token when the input is T'. For each masked token, calculate the cross-entropy loss between the pre-training model prediction distribution and the real word distribution, expressed as: L i =−log P(w i ∣T’); where L i is the cross-entropy loss for position i; Then sum the losses of all masked positions to obtain the loss function L, expressed as: L =∑L i ; Then update the parameters of the pre-training model according to the loss function L through the back propagation algorithm to obtain the model weight after further pre-training; S4. Based on the pre-training model selected in step S3, add an entity extraction sub-model and a relation extraction sub-model downstream of the pre-training model, and use the model weight obtained after further pre-training in step S3 to train the model to obtain a trained knowledge extraction model for network security threat analysis reports; S5. Use the prediction data of the network security threat analysis report original text obtained in step S1 to perform model prediction on the trained knowledge extraction model for network security threat analysis reports, and perform result verification and optimization.

2. The knowledge extraction method for network security intent understanding according to claim 1, characterized in that, The specific implementation method of step S2 comprises the following steps: S2.

1. Replace the IOC token in the training data of the collected network security analysis report original text, replace the hash value, URL, domain name, IP address mentioned in the text with the corresponding token [HASH], [URL], [DOMAIN], [IP] through regular replacement; S2.

2. Then insert a structure token at the beginning of each line of the text, set [h1], [h2], [p] and [tr] to represent the structure of the line in the text, respectively representing the first-level title, the second-level title, the paragraph of the text and the table row. S2.

3. Divide the processed text into a sequence of wordpieces T = {w1, w2, w3,..., wn} using a wordpiece tokenizer, where wn n is the nth wordpiece, and the sequence length does not exceed the maximum length of the pre-trained model. n S2.

4. Encode the sequence of wordpieces T into a sequence of vectors X = {x1, x2, x3,..., xn} using the pre-trained model.

3. The knowledge extraction method for network security intent understanding according to claim 2, characterized in that, The network security analysis report original text exceeding the maximum length of the pre-trained model is first divided into multiple pieces meeting the length requirement in step S2.

1.

4. The knowledge extraction method for network security intent understanding according to claim 3, characterized in that, The specific implementation method of step S4 includes the following steps: S4.

1. Add an entity extraction sub-model downstream of the pre-trained model, use a conditional random field (CRF) as the entity extraction sub-model, and the expression is as follows: ； where x is an input sequence, is the conditional probability distribution of output sequence y given input sequence x; is a characteristic function, λ k is a weight of the characteristic function; is a normalization factor; The input sequence x=BERT(T) is set as the output feature vector sequence of the pre-trained model, y is the BIO label corresponding to each word element, and the normalization factor is obtained by summing all possible output sequences, and the expression is as follows: ； wherein, denotes all possible values of the output label sequence, is the label of the i-th token. S4.

2. Using a feed-forward neural network FFNN as the relation extraction sub-model, obtaining the position of each entity according to the output of the entity extraction sub-model, averaging the feature vectors of all word pieces of each entity x ei and combining each other with the first word piece x [CLS] The corresponding feature vectors are spliced to obtain the input x of the relation extraction sub-model, and the expression is: x = x [CLS] || x ei || x ej ; Then, the relationship type distribution between entities is output through a feedforward neural network, and the expression is as follows: F(x) = max(0, xW1 + b1)W2 + b2 Wherein, W1 and W2 are the weight matrix of the first hidden layer and the weight matrix of the second hidden layer of the feedforward neural network model respectively, and b1 and b2 are the bias term of the first hidden layer and the bias term of the second hidden layer of the feedforward neural network model respectively. S4.

3. Train the entity extraction sub-model using the annotated data set, and set the CRF loss function L CRF , the expression is: L CRF =−logP(y∣x); Setting the FFNN loss function L FFNN , the expression is: L FFNN =−∑ i y i× log(y’ i ); where y i is the true relationship type, y’ i is the FFNN-predicted relationship type; Finally, the weighted average L' is calculated, expressed as: CRF and the FFNN loss function L FFNN Finally, the weighted average L' is calculated, expressed as: L' = λ'L CRF + (1 - λ') L FFNN； Wherein, λ' is the weight value corresponding to the weighted average.

5. An electronic device, comprising: The computer program is executed by the processor to realize the steps of the knowledge extraction method for network security intent understanding according to any one of claims 1-4.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the knowledge extraction method for network security intent understanding according to any one of claims 1-4.

Images