Network threat information labeling method and device, electronic equipment and readable storage medium

Abstract

The application discloses a network threat information labeling method and device, electronic equipment and a readable storage medium, applied to the network security technical field, and the network threat information labeling method comprises the steps of obtaining a target text, wherein the target text carries network threat information; performing sentence splitting on the target text to obtain at least one target sentence; according to each target sentence and a network threat label prediction model, the network threat type of each target sentence is predicted respectively, the network threat information label is obtained, and each target sentence is labeled according to each network threat information label. The application solves the technical problem of low labeling efficiency of network threat information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to a network threat information labeling method and device, electronic equipment and a readable storage medium. BACKGROUND

[0002] As external information resources of direct or potential security threats, network threat information can help security personnel quickly identify malicious threat attacks and respond in time. Therefore, the continuous tracking and analysis of network threat information has become an important task in network security protection. These information is published in the form of unstructured in the website, and key and useful vulnerability and / or attack information can be obtained through information extraction.

[0003] With the rapid development of science and technology, the demand for obtaining network threat information is increasing. At present, professional personnel usually label the text, and manual data labeling requires that the labeling personnel have a solid background in network security knowledge and consume a large amount of time, thereby resulting in low labeling efficiency of network threat information. SUMMARY

[0004] The main purpose of the present application is to provide a network threat information labeling method, device, electronic equipment and readable storage medium, which aims to solve the technical problem of low labeling efficiency of network threat information in the prior art.

[0005] To achieve the above-mentioned purpose, the present application provides a network threat information labeling method, which comprises:

[0006] acquiring a target text, wherein the target text carries network threat information;

[0007] performing sentence splitting on the target text to obtain at least one target sentence;

[0008] According to each target sentence and a network threat label prediction model, the network threat type of each target sentence is predicted to obtain a network threat information label, and each target sentence is labeled according to each network threat information label.

[0009] To achieve the above-mentioned purpose, the present application further provides a network threat information labeling device, which comprises:

[0010] an acquisition module configured to acquire a target text, wherein the target text carries network threat information;

[0011] a splitting module configured to perform sentence splitting on the target text to obtain at least one target sentence;

[0012] The labeling module is configured to predict the network threat type of each target sentence according to the target sentence and a network threat label prediction model, obtain a network threat information label, and label each target sentence according to the network threat information label.

[0013] The present application also provides an electronic device, which comprises a memory, a processor, and a program of the network threat information labeling method stored in the memory and executable on the processor, and the program of the network threat information labeling method can implement the steps of the network threat information labeling method when executed by the processor.

[0014] The present application also provides a computer readable storage medium, which stores a program of the network threat information labeling method, and the program of the network threat information labeling method can implement the steps of the network threat information labeling method when executed by the processor.

[0015] The present application also provides a computer program product, which comprises a computer program, and the computer program can implement the steps of the network threat information labeling method when executed by the processor.

[0016] The present application provides a network threat information labeling method and device, an electronic device, and a readable storage medium. Compared with the method of labeling text by professional personnel, the present application acquires a target text, wherein the target text carries network threat information, splits the target text into at least one target sentence, predicts the network threat type of each target sentence according to the target sentence and a network threat label prediction model, obtains a network threat information label, and labels each target sentence according to the network threat information label. According to the splitting of the sentence and the trained network threat label prediction model, the automatic labeling of the text carrying network threat information is realized, the technical defect that the labeling personnel needs to have a solid network security knowledge background and needs to consume a large amount of time is avoided, and the labeling efficiency of the network threat information is improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0019] Figure 1 A flowchart of a first embodiment of the network threat information labeling method of the present application is shown in FIG. 1.

[0020] Figure 2 A model prediction flowchart of a network threat label prediction model involved in the network threat information labeling method of the embodiments of the present application is shown in FIG. 2.

[0021] Figure 3 A flowchart of a second embodiment of the network threat information labeling method of the present application is shown in FIG. 3.

[0022] Figure 4 A device structure diagram of the network threat information labeling method involved in the embodiments of the present application is shown in FIG. 4.

[0023] Figure 5 A device structure diagram of the hardware running environment of the network threat information labeling method involved in the embodiments of the present application is shown in FIG. 5.

[0024] The purposes, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0025] In order to make the above purposes, features and advantages of the present application more apparent and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0026] Embodiment One

[0027] The embodiments of the present application provide a network threat information labeling method. In a first embodiment of the network threat information labeling method of the present application, referring to FIG. 1, Figure 1 The network threat information labeling method comprises the following steps.

[0028] Step S10, obtaining a target text, wherein the target text carries network threat information.

[0029] As an example, step S10 comprises: obtaining the content entered by a user, and taking the target content carrying network threat information in the content as the target text.

[0030] As an example, step S10 comprises: performing sentence recognition on a target object to obtain the target text carrying network threat information, wherein the target object can be a webpage, a report or other carriers carrying text information.

[0031] Step S20, performing sentence splitting on the target text to obtain at least one target sentence;

[0032] For example, each punctuation symbol in the target text is screened, and each punctuation symbol is used as a splitting symbol to split the target text into at least one target sentence, wherein the punctuation symbol includes at least one of a period, an exclamation mark, and a question mark; and the target sentence includes at least one sentence element, wherein the sentence element is a word and / or a character.

[0033] Step S30, predicting the network threat type of each target sentence according to each target sentence and a network threat label prediction model to obtain a network threat information label, and labeling each target sentence according to each network threat information label.

[0034] As an example, each target sentence is mapped to a corresponding network threat type of each target sentence by a network threat label prediction model, a network threat information label is generated according to the network threat type, and each target sentence is labeled according to each network threat information label.

[0035] As an example, each target sentence is mapped to a corresponding network threat information label of each target sentence by a network threat label prediction model, and each target sentence is labeled according to each network threat information label.

[0036] In step S30, the network threat label prediction model includes a pre-trained language model and a conditional random field model,

[0037] The step of predicting the network threat type of each target sentence according to each target sentence and a network threat label prediction model to obtain a network threat information label includes:

[0038] Step S31, obtaining a preset random inactivation number;

[0039] In this embodiment, it should be noted that the preset random inactivation number is a number of random node inactivation of a model hidden layer that is set in advance.

[0040] It can be understood that, as defined above, the preset random inactivation number is common in model application and model training. When the model is trained, the greater the preset random inactivation number, the longer the training time. However, the model training effect does not increase linearly (that is, the longer the training time, the better the model training effect will not be). Therefore, the preset random inactivation number will not be set too high, and the preset random inactivation number can be 2, 3, or other numbers.

[0041] Step S32, according to the preset random inactivation number, the pre-training language model is used for semantic analysis of each target sentence to obtain a semantic vector;

[0042] For example, the pre-training language model is adjusted according to the preset random inactivation number, and each target sentence is analyzed according to the adjusted pre-training language model to obtain a number of semantic vectors consistent with the preset random inactivation number.

[0043] Step S33, according to the conditional random field model, the probability that each semantic vector belongs to each network threat type is predicted to obtain a prediction result;

[0044] As an example, the conditional random field model is unique, and the probability that each semantic vector belongs to each network threat type is predicted according to the conditional random field model to obtain a prediction result.

[0045] As an example, there are multiple conditional random field models, the number of conditional random field models is consistent with the preset random inactivation number, and the probability that each semantic vector corresponding to each conditional random field model belongs to each network threat type is predicted to obtain a prediction result.

[0046] Step S34, according to the prediction result, the network threat information label corresponding to each target sentence is generated.

[0047] As an example, in the prediction probability, the probability that each semantic vector belongs to each network threat type is selected as the network threat information label.

[0048] As an example, according to the conditional random field model, the network threat information label corresponding to each semantic vector is predicted.

[0049] Specifically, the model parameters of the conditional random field model are obtained, and the transition probability between each two adjacent network threat types is determined according to the model parameters; according to the transition probability and the semantic vector, the probability that each semantic vector corresponding to each semantic vector belongs to each network threat information label is predicted.

[0050] Optionally, the step of predicting the probability that each semantic vector corresponding to each semantic vector belongs to each network threat information label according to the transition probability and the semantic vector can specifically include:

[0051]

[0052] wherein, S g(x, Z) represents the probability that the target statement x output by the g-th conditional random field model belongs to the network threat information label Z. For the tag z i Transfer to z i+1 The probability, U x [w i ][z i [W is the word w in the target statement x] i This belongs to the network threat information tag z i The probability of.

[0053] In step S32, the step of performing semantic analysis on each target statement using the pre-trained language model based on the preset number of random deactivation attempts to obtain semantic vectors includes:

[0054] Step A10: Randomly deactivate nodes in the hidden layer of the pre-trained language model according to the preset number of random deactivation attempts;

[0055] As an example, the pre-trained speech model has multiple hidden layers, and the number of hidden layers is consistent with the preset number of random deactivation times. Random node deactivation is performed on each hidden layer to obtain each random node deactivated hidden layer.

[0056] Specifically, the hidden layer includes a self-attention mechanism layer and a feedforward layer, and random nodes are deactivated in the self-attention mechanism layer and / or the feedforward layer.

[0057] Step A20: Semantic analysis is performed on each target statement using the pre-trained language model after random node deactivation to obtain semantic vectors.

[0058] As an example, semantic analysis of each target statement is performed using the pre-trained language model under each hidden layer to obtain semantic vectors.

[0059] As an example, random node deactivation is performed on the hidden layer of the pre-trained language model. Semantic analysis is then performed on each target statement using the pre-trained language model after random node deactivation to obtain semantic vectors. The total number of random node deactivations in the hidden layer is accumulated. It is then determined whether the total number of random node deactivations is less than or equal to a preset number of random node deactivations. If the total number of random node deactivations is less than or equal to the preset number of random node deactivations, the process returns to the step of performing random node deactivation on the hidden layer of the pre-trained language model and subsequent steps, until the total number of random node deactivations is greater than the preset number of random node deactivations.

[0060] Specifically, the hidden layer of the pre-trained language model comprises an embedding layer, a self-attention mechanism layer, a normalization layer, a feedforward layer, a summation and normalization layer, and a fully connected layer. According to the preset number of random deactivation, the feedforward layer and the self-attention mechanism layer are respectively subjected to node random deactivation to obtain a node random deactivation feedforward layer and a node random deactivation self-attention mechanism layer. Each target sentence is input into the pre-trained language model, and each target sentence is processed in sequence by the embedding layer, the node random deactivation self-attention mechanism layer, the normalization layer, the random deactivation feedforward layer, and the summation and normalization layer to obtain a processing vector. The processing vector is subjected to feature integration by a preset number of fully connected layers to obtain a semantic vector, wherein the preset number can be 2.

[0061] As an example, refer to Figure 2 , Figure 2 FIG. 1 is a schematic diagram of a model prediction process of a network threat label prediction model involved in a network threat information labeling method in an embodiment of the present application. Figure 2 The network threat label prediction model comprises an embedding layer, a self-attention mechanism layer, a normalization layer, a feedforward layer, a summation and normalization layer, and a conditional random field model. A target text (x in the figure) is input into the network threat label prediction model, and the target text is processed in sequence by the embedding layer, the self-attention mechanism layer, the normalization layer, the feedforward layer, the summation and normalization layer, and the conditional random field model in the network threat label prediction model to obtain probabilities that the target text belongs to each network threat information label. The network threat information label corresponding to the maximum probability in each probability is taken as the final label.

[0062] The embodiment of the present application provides a network threat information labeling method. Compared with a method of labeling a text by a professional, the embodiment of the present application acquires a target text, wherein the target text carries network threat information; performs sentence splitting on the target text to obtain at least one target sentence; predicts a network threat type of each target sentence according to each target sentence and a network threat label prediction model to obtain a network threat information label, and labels each target sentence according to each network threat information label. The network threat information labeling method realizes automatic labeling of a text carrying network threat information, avoids the technical defect that a labeling personnel needs to have a solid network security knowledge background and needs to spend a large amount of time, and thus improves the labeling efficiency of network threat information.

[0063] Embodiment two

[0064] Further, refer to Figure 3, based on the first embodiment of the present application, in another embodiment of the present application, the same or similar content as the above embodiment one can refer to the above introduction, and the subsequent will not be described. On this basis, wherein, in step S30, before the step of predicting the network threat type of each target sentence according to each target sentence and network threat label prediction model, respectively, obtaining network threat information label, still includes:

[0065] Step B10, obtaining training samples, real labels corresponding to the training samples, a predetermined random inactivation number and a network threat label prediction model to be trained, wherein the network threat label prediction model to be trained includes a language model to be trained and a conditional random field model to be trained;

[0066] In this embodiment, it should be noted that the training sample is at least one sentence containing network threat information used for model training of the network threat label prediction model to be trained. The real label is the real network threat type information of the training sample. The network threat label prediction model to be trained is an untrained network threat label prediction model.

[0067] Exemplarily, the preset random inactivation number set by the user or the system has, training samples, real labels corresponding to the training samples and a network threat label prediction model to be trained.

[0068] Step B20, according to the preset random inactivation number, the semantic analysis of each target sentence is carried out through the language model to be trained, and the training semantic vector is obtained;

[0069] Step B30, according to the conditional random field model to be trained, the probability that each training semantic vector belongs to each network threat information label is predicted, and the training prediction result is obtained;

[0070] Exemplarily, the specific implementation steps of steps B20 to B30 can refer to the specific implementation content of steps S32 to S34 and steps A10 to A20 in the above text, and will not be described here.

[0071] Step B40, according to the training prediction result, the preset random inactivation number and the real label, the model loss of the network threat label prediction model to be trained is constructed;

[0072] Step B50, according to the model loss, the network threat label prediction model to be trained is iteratively optimized, and the network threat label prediction model is obtained.

[0073] Exemplarily, the step B40 to the step B50 comprises: constructing a model loss of the network threat label prediction model to be trained according to the training prediction result, the preset random inactivation number and the real label; judging whether the model loss converges, if the model loss converges, taking the network threat label prediction model to be trained as the network threat label prediction model; if the model loss does not converge, returning to execute the step of obtaining the training sample, the real label corresponding to the training sample, the preset random inactivation number and the network threat label prediction model to be trained and the subsequent steps until the model loss converges.

[0074] In the step B10, the obtaining the training sample and the real label corresponding to the training sample comprises:

[0075] In the step B11, the annual report and the network page information of each security vendor are parsed to obtain the original sample.

[0076] In the embodiment, it should be noted that the original sample can be an APT (Advanced Persistent Threat) report.

[0077] As an example, after the step B10, the original sample is subjected to data cleaning.

[0078] In the step B12, the original sample is subjected to sample screening and sample content preprocessing to obtain a processed sample.

[0079] It can be understood that the directly obtained original sample, for example, the APT report, because it involves detailed information such as attack events and attack analysis, is often long in length. If the original sample is directly subjected to sentence splitting, and the model is trained according to each sentence obtained by splitting, the efficiency of model training is easily low.

[0080] Exemplarily, the original sample is subjected to sample screening to obtain a screened sample, and the screened sample is subjected to deduplication processing to obtain a processed sample.

[0081] As an example, the text length of each text in the original sample is obtained, and each text in the original sample whose text length is greater than or equal to a preset length threshold is taken as a length screening sample, where the preset length threshold is a length screening critical value that is preset to ensure the content richness of the original sample, and the preset length threshold can be 400 bytes. The text keyword of each text in the length screening sample is extracted, and each text in the length screening sample whose keyword frequency of the text keyword is greater than a preset frequency threshold and whose keyword frequency is greater than a preset frequency threshold is taken as a screening sample, where the text keyword is a keyword related to the network security field, the preset frequency threshold is a keyword frequency critical value that is preset to determine whether the keyword frequency of the text meets the network threat information richness requirement, and the preset frequency threshold is a keyword frequency critical value that is preset to determine whether the keyword frequency of the text meets the network threat information richness requirement.

[0082] Optionally, the step of taking, in each text in the length screening sample, the text keyword whose keyword frequency is greater than a preset frequency threshold and whose keyword frequency is greater than a preset frequency threshold as a screening sample can specifically include:

[0083] len(doc∩V)>a

[0084] wherein len(doc∩V) is the keyword frequency of the text keyword, V is the text keyword, and a is the preset frequency threshold.

[0085]

[0086] wherein p(doc∩V) is the keyword frequency of the text keyword, sum(w|w(∈V∩doc)) is the total number of occurrences of the text keyword, len(doc) is the text length of the length screening sample, and b is the preset frequency threshold.

[0087] As an example, the words contained in each text in the screening sample and the text publishing time corresponding to each text in the screening sample are obtained, and each text in the screening sample whose word repetition rate is less than or equal to a preset repetition rate threshold and / or the time interval between the text publishing times is greater than or equal to a preset time interval is taken as a processing sample, where the preset repetition rate threshold is a word repetition rate critical value that is preset to determine whether the screening sample has more repeated content between texts, and the preset time interval is a text publishing time critical value that is preset to determine whether the screening sample has a shorter publishing interval between texts. For example, the processing sample whose word repetition rate is less than or equal to 0.03 in each text in the screening sample, and / or the processing sample whose time interval between the text publishing times is greater than or equal to 7 days.

[0088] It can be understood that, by screening and deduplicating the original samples, the obtained processing samples have rich network security knowledge and reduced processing amount. Since the processing samples are used for model training, the efficiency of model training is improved under the premise of ensuring the accuracy of model training.

[0089] Step B13, performing sentence splitting on the processing samples to obtain the training samples;

[0090] By way of example, the implementation steps of step B13 can refer to the implementation content of step S20 described above, and will not be described here.

[0091] Step B14, determining the true label corresponding to each training sample according to each training sample.

[0092] It can be understood that, at present, the true label of each sample needs to be manually labeled when model training is performed. Artificial data labeling requires that the labeling personnel have a solid network security knowledge background and consume a large amount of time, thereby causing the time required for determining the true label to be relatively long, and further causing the efficiency of model training to be relatively low.

[0093] Through the above processing of the original samples, the volume of the text that needs to be labeled is reduced to a certain extent, thereby reducing the time required for determining the true label, and further improving the efficiency of model training.

[0094] In step B14, the true label includes entity boundary, entity category, relationship type, and entity position,

[0095] The step of determining the true label corresponding to each training sample according to each training sample includes:

[0096] Step C10, determining at least one entity in each training sample;

[0097] In this embodiment, it should be noted that the entity is a cyberspace security entity, which is used to describe the basic component unit in the field of network security.

[0098] Step C20, obtaining a preset labeling mode, and determining the entity boundary of each entity according to the preset labeling mode;

[0099] In the embodiment, it should be noted that the preset annotation mode is a mode for annotating the entity boundary, and the preset annotation mode can be a BIOES (begin inside outside end single) mode. B represents that each word in each training sample is at the beginning position of each entity, I represents that each word is inside each entity, E represents that each word is at the end position of each entity, S represents that each word is a single entity, and O represents that each word is a non-entity.

[0100] Step C30, performing category analysis on each entity to obtain an entity category.

[0101] In the embodiment, it should be noted that the entity category at least includes one of Organization, Location, Software, Malware, Indicator, Vulnerability, Course-of-action, Tool, Attack-pattern, Industry and Technique.

[0102] Step C40, taking the position information corresponding to each entity as the entity position.

[0103] For example, the position information of each entity in a relationship triple is determined, and the position information is taken as the entity position. The position information at least includes one of a head entity and a tail entity.

[0104] Step C50, determining the relationship between each entity, to obtain the relationship type.

[0105] In the embodiment, it should be noted that the relationship type at least includes one of uses, comes-from, has-vulnerability, has-product, related-to, include, belong, track and has-target.

[0106] In step B30, the step of constructing the model loss of the network threat label prediction model to be trained according to the training prediction result, the preset random inactivation number and the real label includes:

[0107] Step B31, constructing the conditional random field model loss according to the training prediction result, the preset random inactivation number and the real label.

[0108] According to the probability that each training semantic vector belongs to each network threat information label in the training prediction result, the conditional random field model loss is constructed according to the probability that each training semantic vector belongs to each network threat information label, the preset random deactivation number and the real label.

[0109] Optionally, the step of constructing the conditional random field model loss according to the probability that each training semantic vector belongs to each network threat information label, the preset random deactivation number and the real label can specifically include:

[0110]

[0111] wherein, L (CRF) is the conditional random field model loss, k is the preset random deactivation number, S g (x 1 , Z1) is the probability that the training semantic vector x 1 output by the gth conditional random field model to be trained belongs to the network threat information label Z1, S g (x 1 , Z2) is the probability that the training semantic vector x 1 output by the gth conditional random field model to be trained belongs to the network threat information label Z2, S g (x 1 , Z N ) is the probability that the training semantic vector x 1 output by the gth conditional random field model to be trained belongs to the network threat information label Z N .

[0112] Step B32, constructing the relative entropy loss corresponding to the preset random deactivation number according to the training prediction result and the preset random deactivation number.

[0113] Optionally, in the training prediction result, the target training prediction result with the maximum probability and the target adjacent training prediction result adjacent to the target training prediction result are selected, the first divergence value of the target training prediction result with respect to the target adjacent training prediction result and the second divergence value of the target adjacent training prediction result with respect to the target training prediction result are determined according to the target training prediction result and the target adjacent training prediction result, and the relative entropy loss corresponding to the preset random deactivation number is constructed according to the first divergence value, the second divergence value and the preset random deactivation number.

[0114] Optionally, the step of constructing the relative entropy loss corresponding to the preset random deactivation number according to the first divergence value, the second divergence value and the preset random deactivation number can specifically include:

[0115]

[0116] wherein, L (KL) is the relative entropy loss corresponding to the preset random inactivation number, k is the preset random inactivation number, KL(P * j is the first divergence value of the target training prediction result with respect to the target adjacent training prediction result, KL(P * j+1 ) is the second divergence value of the target adjacent training prediction result with respect to the target training prediction result. * j+1 * j

[0117] Optionally, the step of determining the first divergence value of the target training prediction result with respect to the target adjacent training prediction result according to the target training prediction result and the target adjacent training prediction result can specifically include:

[0118]

[0119] wherein, KL(P * j |P * j+1 ) is the first divergence value of the target training prediction result with respect to the target adjacent training prediction result, P * j is the target training prediction result, P * j+1 is the target adjacent training prediction result.

[0120] Optionally, the step of determining the second divergence value of the target adjacent training prediction result with respect to the target training prediction result according to the target training prediction result and the target adjacent training prediction result can specifically include:

[0121]

[0122] wherein, KL(P * j+1 |P * j ) is the second divergence value of the target adjacent training prediction result with respect to the target training prediction result, P * j is the target training prediction result, P * j+1 is the target adjacent training prediction result.

[0123] ​​Step B33, integrating the conditional random field model loss and the relative entropy loss as a model loss of the network threat label prediction model to be trained.

[0124] Exemplarily, a sum of the conditional random field model loss and the relative entropy loss is taken as a model loss of the network threat label prediction model to be trained.

[0125] Optionally, the step of integrating the conditional random field model loss and the relative entropy loss as a model loss of the network threat label prediction model to be trained can specifically include:

[0126] L = L (KL) + L (CRF)

[0127] wherein L is a model loss of the network threat label prediction model to be trained, L (KL) is the relative entropy loss, and L (CRF) is the conditional random field model loss.

[0128] The embodiment of the present application provides a network threat information labeling method, and the embodiment of the present application obtains a training sample, a real label corresponding to the training sample, a preset random inactivation number and a network threat label prediction model to be trained, wherein the network threat label prediction model to be trained includes a language model to be trained and a conditional random field model to be trained; according to the preset random inactivation number, the language model to be trained is used to perform semantic analysis on each target sentence to obtain a training semantic vector; according to the conditional random field model to be trained, a probability that each training semantic vector belongs to each network threat information label is predicted to obtain a training prediction result; according to the training prediction result, the preset random inactivation number and the real label, a model loss of the network threat label prediction model to be trained is constructed; and according to the model loss, the network threat label prediction model to be trained is iteratively optimized to obtain the network threat label prediction model, so that the training efficiency is high when the network threat label prediction model to be trained is trained, and the model prediction accuracy of the network threat label prediction model obtained by training is high.

[0129] Embodiment three

[0130] The embodiment of the present application also provides a network threat information labeling device, and the network threat information labeling device is referred to Figure 4 , and the network threat information labeling device includes:

[0131] An acquisition module is configured to acquire a target text, wherein the target text carries network threat information.

[0132] A splitting module is configured to split the target text into at least one target sentence.

[0133] The labeling module is configured to predict, according to each target sentence and a network threat label prediction model, a network threat type of each target sentence, obtain a network threat information label, and label each target sentence according to the network threat information label.

[0134] Optionally, the network threat label prediction model comprises a pre-trained language model and a conditional random field model, and the labeling module is further configured to:

[0135] acquire a preset random deactivation number;

[0136] perform semantic analysis on each target sentence by using the pre-trained language model according to the preset random deactivation number, and obtain a semantic vector;

[0137] predict, according to the conditional random field model, a probability that each semantic vector belongs to each network threat type, and obtain a prediction result;

[0138] generate a network threat information label corresponding to each target sentence according to the prediction result.

[0139] Optionally, the labeling module is further configured to:

[0140] randomly deactivate hidden layers of the pre-trained language model according to the preset random deactivation number;

[0141] perform semantic analysis on each target sentence by using the pre-trained language model after random deactivation, and obtain a semantic vector.

[0142] Optionally, before the step of predicting, according to each target sentence and a network threat label prediction model, a network threat type of each target sentence, and obtaining a network threat information label, the network threat information labeling apparatus is further configured to:

[0143] acquire a training sample, a real label corresponding to the training sample, a preset random deactivation number, and a network threat label prediction model to be trained, wherein the network threat label prediction model to be trained comprises a language model to be trained and a conditional random field model to be trained;

[0144] perform semantic analysis on each target sentence by using the language model to be trained according to the preset random deactivation number, and obtain a training semantic vector;

[0145] predict, according to the conditional random field model to be trained, a probability that each training semantic vector belongs to each network threat information label, and obtain a training prediction result;

[0146] According to the training prediction result, the preset random inactivation number and the real label, a model loss of the network threat label prediction model to be trained is constructed;

[0147] According to the model loss, the network threat label prediction model to be trained is iteratively optimized to obtain the network threat label prediction model.

[0148] Optionally, the network threat information labeling apparatus is further configured to:

[0149] Raw samples are obtained by parsing annual reports and network page information of each security vendor;

[0150] The raw samples are subjected to sample screening and sample content preprocessing to obtain processed samples;

[0151] The processed samples are subjected to sentence splitting to obtain the training samples;

[0152] According to each training sample, a real label corresponding to each training sample is determined.

[0153] Optionally, the real label comprises entity boundary, entity category, relationship type and entity position, and the network threat information labeling apparatus is further configured to:

[0154] At least one entity is determined in each training sample;

[0155] A preset labeling mode is obtained, and entity boundaries of each entity are determined according to the preset labeling mode;

[0156] Each entity is subjected to category analysis to obtain an entity category;

[0157] Position information corresponding to each entity is taken as the entity position;

[0158] Relationships between each entity are determined to obtain the relationship type.

[0159] Optionally, the network threat information labeling apparatus is further configured to:

[0160] According to the training prediction result, the preset random inactivation number and the real label, a model loss of the network threat label prediction model to be trained is constructed;

[0161] According to the training prediction result and the preset random inactivation number, a relative entropy loss corresponding to the preset random inactivation number is constructed;

[0162] The conditional random field model loss and the relative entropy loss are integrated into the model loss of the network threat label prediction model to be trained.

[0163] The network threat information labeling device provided in the application adopts the network threat information labeling method in the above embodiment, and solves the technical problem of low labeling efficiency of network threat information. Compared with the prior art, the network threat information labeling device provided in the embodiment of the application has the same beneficial effects as the network threat information labeling method provided in the above embodiment, and other technical features in the network threat information labeling device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0164] Embodiment four

[0165] The embodiment of the application provides an electronic device, which comprises at least one processor and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the network threat information labeling method in the above embodiment.

[0166] Reference will be made to Figure 5 , which shows a structural schematic diagram of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic device in the embodiments of the present disclosure can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Personal Computers), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 5 The electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0167] As shown in Figure 5 , the electronic device can include a processing device (such as a central processor, a graphics processor, etc.), which can perform various appropriate actions and processes according to programs stored in a ROM (Read-Only Memory) or loaded into a RAM (Random Access Memory) from a storage device. In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0168] Generally, the following systems can be connected to the I / O interface: input devices including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and communication devices. The communication devices can allow the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the electronic device is illustrated as having various systems, it is understood that all of the illustrated systems are not required to implement or have. More or less systems can be implemented or have instead.

[0169] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device, or installed from the storage device, or installed from the ROM. When the computer program is executed by the processing device, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0170] The electronic device provided by the present application adopts the network threat information labeling method in the above-mentioned embodiments, and solves the technical problem of low labeling efficiency of network threat information. Compared with the prior art, the electronic device provided by the embodiments of the present application has the same beneficial effects as the network threat information labeling method provided by the above-mentioned embodiments, and other technical features in the electronic device are the same as the features disclosed in the above-mentioned embodiments, which will not be repeated here.

[0171] It should be understood that parts of the present disclosure can be realized by hardware, software, firmware or a combination thereof. In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0172] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0173] Embodiment five

[0174] The present embodiment provides a computer readable storage medium having stored thereon computer readable program instructions for performing the network threat information labeling method of the above-mentioned embodiments.

[0175] The computer readable storage medium provided by the embodiments of the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Erasable Programmable Read Only Memory), or flash memory, an optical fiber, a CD-ROM (compact disc read-only memory), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the embodiments, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination thereof.

[0176] The above computer readable storage medium can be contained in an electronic device, or can exist separately without being assembled into an electronic device.

[0177] The above computer readable storage medium carries one or more programs, when the one or more programs are executed by an electronic device, the electronic device is caused to: acquire a target text, wherein the target text carries network threat information; perform sentence splitting on the target text to obtain at least one target sentence; predict the network threat type of each target sentence according to each target sentence and a network threat label prediction model, obtain a network threat information label, and label each target sentence according to each network threat information label.

[0178] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0179] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block in the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by special-purpose hardware-based systems that perform the specified functions or operations, or combinations of hardware and computer instructions.

[0180] The modules involved in the embodiments of the present disclosure can be implemented in the manner of software or hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0181] The computer readable storage medium provided by the present application stores computer readable program instructions for executing the network threat information labeling method described above, and solves the technical problem of low labeling efficiency of network threat information. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the embodiments of the present application are the same as those of the network threat information labeling method provided by the above embodiments, and are not described here.

[0182] Embodiment six

[0183] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the network threat information labeling method as described above.

[0184] The computer program product provided by the application solves the technical problem of low labeling efficiency of network threat information. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiments of the application are the same as those of the network threat information labeling method provided by the above-mentioned embodiments, and are not described here.

[0185] The above is only the preferred embodiment of the application, and does not limit the patent scope of the application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent processing scope of the application.

Claims

1. A method for labeling network threat information, characterized in that, The network threat information annotation method includes: Obtain target text, wherein the target text carries network threat information; The target text is split into sentences to obtain at least one target sentence; Based on the target statements and the network threat label prediction model, the network threat type of each target statement is predicted to obtain network threat information labels, and each target statement is labeled according to the network threat information labels; wherein, the network threat label prediction model includes a pre-trained language model after random node deactivation, which is used to perform semantic analysis on each target statement to obtain semantic vectors; The network threat label prediction model is obtained by training the network threat label prediction model to be trained using training samples. The steps for obtaining training samples include: parsing the annual reports and web page information of various security vendors to obtain raw samples. The texts in the original sample whose length is greater than or equal to a preset length threshold are used as length-filtered samples. Extract the text keywords of each text in the length-filtered sample, and determine the filter samples in the length-filtered sample in which the number of the text keywords in each text is greater than a preset number threshold and the keyword frequency is greater than a preset frequency threshold; Obtain the words contained in each text in the filtered sample and the text publication time corresponding to each text in the filtered sample, and select the processing samples from each text in the filtered sample whose word repetition rate is less than or equal to a preset repetition rate threshold and / or whose time interval between each text publication time is greater than or equal to a preset time interval; The processed sample is split into sentences to obtain the training sample.

2. The network threat information labeling method as described in claim 1, characterized in that, The network threat label prediction model includes a pre-trained language model and a conditional random field model. The step of predicting the network threat type of each target statement based on the target statement and the network threat label prediction model to obtain the network threat information label includes: Get the preset number of random deactivation attempts; Based on the preset number of random deactivation attempts, semantic analysis is performed on each of the target statements using the pre-trained language model to obtain semantic vectors; Based on the conditional random field model, the probability that each semantic vector belongs to each network threat type is predicted, and the prediction result is obtained. Based on the prediction results, network threat information tags are generated for each of the target statements.

3. The network threat information labeling method as described in claim 2, characterized in that, The step of performing semantic analysis on each target statement using the pre-trained language model based on the preset number of random deactivation attempts to obtain semantic vectors includes: Based on the preset number of random deactivation attempts, random nodes are deactivated in the hidden layer of the pre-trained language model; Semantic vectors are obtained by performing semantic analysis on each target statement using the pre-trained language model after random node deactivation.

4. The network threat information labeling method as described in claim 1, characterized in that, Before the step of predicting the network threat type of each target statement based on the target statement and the network threat label prediction model to obtain the network threat information label, the method further includes: Acquire training samples, the real labels corresponding to the training samples, a preset number of random deactivation times, and a network threat label prediction model to be trained, wherein the network threat label prediction model to be trained includes a language model to be trained and a conditional random field model to be trained. Based on the preset number of random deactivation attempts, semantic analysis is performed on each of the target sentences using the language model to be trained to obtain training semantic vectors; Based on the conditional random field model to be trained, predict the probability that each training semantic vector belongs to each network threat information label, and obtain the training prediction result; Based on the training prediction results, the preset random deactivation times, and the real labels, construct the model loss of the network threat label prediction model to be trained; Based on the model loss, the network threat label prediction model to be trained is iteratively optimized to obtain the network threat label prediction model.

5. The network threat information labeling method as described in claim 4, characterized in that, The step of obtaining the real labels corresponding to the training samples includes: Based on each training sample, determine the true label corresponding to each training sample.

6. The network threat information annotation method as described in claim 5, characterized in that, The real labels include entity boundaries, entity category, relationship type, and entity location. The step of determining the true label corresponding to each training sample based on each training sample includes: At least one entity is identified in each of the training samples; Obtain a preset annotation pattern, and determine the entity boundary of each entity according to the preset annotation pattern; Perform category analysis on each entity to obtain entity categories; The location information corresponding to each entity is used as the entity location; Determine the pairwise relationships between each of the entities to obtain the relationship type.

7. The network threat information annotation method as described in claim 4, characterized in that, The step of constructing the model loss of the network threat label prediction model to be trained based on the training prediction results, the preset number of random deactivation attempts, and the real labels includes: Based on the training prediction results, the preset number of random deactivations, and the true labels, the conditional random field model loss is constructed. Construct the relative entropy loss corresponding to the preset number of random inactivations based on the training prediction results and the preset number of random inactivations; The conditional random field model loss and the relative entropy loss are integrated into the model loss of the network threat label prediction model to be trained.

8. A network threat information labeling device, characterized in that, The network threat information labeling device includes: An acquisition module is used to acquire target text, wherein the target text carries network threat information; The splitting module is used to split the target text into sentences to obtain at least one target sentence. The annotation module is used to predict the network threat type of each target statement based on the target statements and the network threat label prediction model, obtain network threat information labels, and annotate each target statement according to the network threat information labels; wherein, the network threat label prediction model includes a pre-trained language model after random node inactivation, which is used to perform semantic analysis on each target statement to obtain semantic vectors; the network threat label prediction model is obtained by training the network threat label prediction model to be trained using training samples. The annotation module is also used to parse the annual reports and web page information of various security vendors to obtain raw samples; The texts in the original sample whose length is greater than or equal to a preset length threshold are used as length-filtered samples. Extract the text keywords of each text in the length-filtered sample, and determine the filter samples in the length-filtered sample in which the number of the text keywords in each text is greater than a preset number threshold and the keyword frequency is greater than a preset frequency threshold; Obtain the words contained in each text in the filtered sample and the text publication time corresponding to each text in the filtered sample, and select the processing samples from each text in the filtered sample whose word repetition rate is less than or equal to a preset repetition rate threshold and / or whose time interval between each text publication time is greater than or equal to a preset time interval; The processed sample is split into sentences to obtain the training sample.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the network threat information labeling method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for implementing a network threat information labeling method, which is executed by a processor to implement the steps of the network threat information labeling method as described in any one of claims 1 to 7.

Images