Network security entity classification method, network security entity construction method, network security entity classification device, network security entity construction device, equipment and medium

Abstract

The invention relates to the technical field of network security, and discloses a network security entity classification method, a construction method, a device, equipment and a medium, and the network security entity classification method comprises the steps: obtaining to-be-classified entity data; the to-be-classified entity data are input into a fusion model, a fusion feature vector output by the fusion model is determined, and the fusion model is a model formed by sequentially combining a BERT-large language model, a plurality of neural network models and an attention mechanism; inputting the fusion feature vector into an activation function; and based on the output value of the activation function, determining whether the to-be-classified entity data is the entity data in the network security field. According to the method, the BERT-large language model, the multiple neural network models and the fusion model combined by the attention mechanism are introduced, so that efficient and accurate classification of the network security entities is realized.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a network security entity classification method, construction method, device, equipment and medium. Background Art

[0002] As an important information storage, representation and application tool, knowledge graph can organize, manage and apply massive data in a unified manner. By integrating and analyzing various security information through knowledge graph, the overall situation of network security can be fully presented, and potential security threats and risk points can be quickly discovered. For example, by correlating and analyzing attack tools, vulnerabilities, affected systems and other information, large-scale network attacks that may occur can be predicted.

[0003] In the construction of the network security knowledge graph, entity extraction is the core task and important link of knowledge extraction. Entity extraction is to identify various entities in the network security field from preprocessed text data, such as network attack tools, security vulnerabilities, and network protocols. At present, after entity extraction, it is mainly determined manually whether the acquired entity is related to the network security field, and the efficiency and accuracy of entity classification are poor. Summary of the invention

[0004] In view of this, the present invention provides a network security entity classification method, construction method, device, equipment and medium to solve the problem that it is difficult to accurately and efficiently determine whether the acquired entity is an entity related to the network security field.

[0005] In a first aspect, the present invention provides a method for classifying network security entities, the method comprising: obtaining entity data to be classified; inputting the entity data to be classified into a fusion model, and determining a fusion feature vector output by the fusion model, wherein the fusion model is a model in which a BERT-large language model, multiple neural network models, and an attention mechanism are sequentially combined; inputting the fusion feature vector into an activation function; and determining whether the entity data to be classified is entity data in the field of network security based on the output value of the activation function.

[0006] The network security entity classification method provided in this embodiment, after obtaining the entity data to be classified, inputs the entity data to be classified into a fusion model formed by a combination of a BERT-large language model, multiple neural network models and an attention mechanism to obtain a fusion feature vector, and then inputs the fusion feature vector into an activation function, and determines whether the entity data to be classified is entity data in the network security field based on the output value of the activation function, which can perform network security entity classification relatively quickly and accurately. This embodiment determines the classification result from entity features extracted from multiple dimensions, enhances the entity classification capability of the fusion model, improves the accuracy of the classification result, and also reduces the calculation dimension by focusing only on the current task features, thereby improving the classification efficiency.

[0007] In an optional embodiment, the multiple neural network models include a deep pyramid convolutional neural network, a bidirectional long short-term memory network, a first deep neural network, a second deep neural network and a third deep neural network; the input end of the deep pyramid convolutional neural network, the input end of the bidirectional long short-term memory network and the input end of the first deep neural network are all connected to the output end of the BERT-large language model; the output end of the deep pyramid convolutional neural network, the output end of the bidirectional long short-term memory network and the output end of the first deep neural network are all connected to the input end of the second deep neural network; the output end of the second deep neural network is connected to the input end of the third deep neural network, and the output end of the third deep neural network is connected to the input end of the attention mechanism.

[0008] In an optional embodiment, the fusion model further includes a batch normalization layer, an input end of the batch normalization layer is connected to an output end of the second deep neural network, and an output end of the batch normalization layer is connected to an input end of the third deep neural network.

[0009] In this embodiment, the batch normalization layer can improve the training effect of the deep neural network by standardizing the input of each layer, regularizing the data into a uniform interval, reducing internal covariate shift, reducing the divergence of data, reducing the learning difficulty of the network, helping to accelerate training and improve the stability of the model.

[0010] In an optional implementation, the activation function is:

[0011]

[0012] Where x represents a vector containing K real numbers, softmax(x) i Represents x i Vector of output values.

[0013] In an optional embodiment, before inputting the entity data to be classified into the fusion model and determining the fusion feature vector output by the fusion model, the method also includes: constructing a fusion model based on an initial BERT-large language model, multiple initial neural network models and an initial attention mechanism.

[0014] In an optional implementation, a fusion model is constructed based on an initial BERT-large language model, multiple initial neural network models, and an initial attention mechanism, including: creating a pre-training data set and a network security entity data set; obtaining an initial BERT-large language model, multiple initial neural network models, and an initial attention mechanism; training the initial BERT-large language model based on the pre-training data set to obtain an updated initial BERT-large language model; combining the updated initial BERT-large language model, multiple initial neural network models, and the initial attention mechanism to form an initial fusion model; and training the initial fusion model based on the network security entity data set to obtain a fusion model.

[0015] In this embodiment, by fine-tuning the BERT-large language model twice, the model's ability to represent network security-related entity information can be enhanced, and different entities can be better represented and distinguished.

[0016] In a second aspect, the present invention provides a method for constructing a network security knowledge graph, the method comprising: obtaining entity data to be classified; inputting the entity data to be classified into a fusion model, and determining a fusion feature vector output by the fusion model, wherein the fusion model is a model in which a BERT-large language model, multiple neural network models, and an attention mechanism are sequentially combined; inputting the fusion feature vector into an activation function; based on the output value of the activation function, determining whether the entity data to be classified is entity data in the field of network security; when the entity data to be classified is entity data in the field of network security, adding the entity data to be classified to an entity library of the network security knowledge graph.

[0017] In a third aspect, the present invention provides a network security entity classification device, the device comprising: an acquisition module, used to acquire entity data to be classified; a first determination module, used to input the entity data to be classified into a fusion model, and determine the fusion feature vector output by the fusion model, wherein the fusion model is a model in which a BERT-large language model, a plurality of neural network models and an attention mechanism are sequentially combined; an input module, used to input the fusion feature vector into an activation function; and a second determination module, used to determine whether the entity data to be classified is entity data in the field of network security based on the output value of the activation function.

[0018] In a fourth aspect, the present invention provides a device for constructing a network security knowledge graph, the device comprising: an acquisition module for acquiring entity data to be classified; a first determination module for inputting the entity data to be classified into a fusion model to determine a fusion feature vector output by the fusion model, wherein the fusion model is a model in which a BERT-large language model, multiple neural network models, and an attention mechanism are sequentially combined; an input module for inputting the fusion feature vector into an activation function; a second determination module for determining whether the entity data to be classified is entity data in the field of network security based on the output value of the activation function; and an adding module for adding the entity data to be classified to the entity library of the network security knowledge graph when the entity data to be classified is entity data in the field of network security.

[0019] In a fifth aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the network security entity classification method of the above-mentioned first aspect or any corresponding embodiment thereof, or executes the network security knowledge graph construction method of the above-mentioned second aspect or any corresponding embodiment thereof by executing the computer instructions.

[0020] In a sixth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the network security entity classification method of the above-mentioned first aspect or any corresponding embodiment thereof, or to execute the network security knowledge graph construction method of the above-mentioned second aspect or any corresponding embodiment thereof.

[0021] In the seventh aspect, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute the network security entity classification method of the above-mentioned first aspect or any corresponding embodiment, or execute the network security knowledge graph construction method of the above-mentioned second aspect or any corresponding embodiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 is a schematic diagram of a flow chart of a network security entity classification method according to an embodiment of the present invention;

[0024] Figure 2is a flow chart of another network security entity classification method according to an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of a result of a fusion model according to an embodiment of the present invention:

[0026] Figure 4 : is a schematic diagram of a process for determining a pre-training data set and a network security entity data set according to an embodiment of the present invention:

[0027] Figure 5 It is a flowchart of a method for constructing a network security knowledge graph according to an embodiment of the present invention;

[0028] Figure 6 It is a flowchart of another method for constructing a network security knowledge graph according to an embodiment of the present invention;

[0029] Figure 7 is a structural block diagram of a network security entity classification device according to an embodiment of the present invention;

[0030] Figure 8 is a structural block diagram of a network security knowledge graph construction device according to an embodiment of the present invention;

[0031] Fig. 9 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0033] In the Natural Language Processing (NLP) technology center, the construction of knowledge graphs plays a very important role in the development of artificial intelligence. In the construction of knowledge graphs, entity extraction is the core task and important link of knowledge extraction.

[0034] However, how to filter domain-related entities and how to correctly classify domain entities during the knowledge graph construction process is still a bottleneck problem in the autonomous expansion and construction of knowledge graphs. In particular, in the construction of network security knowledge graphs, there is a lack of accurate and efficient methods for judging whether newly acquired entity data is related to network security in the process of autonomous graph construction.

[0035] In view of this, the present invention provides a network security entity classification method, which realizes efficient and accurate classification of network security entities by introducing a fusion model of the BERT-large language model, multiple neural network models and an attention mechanism.

[0036] According to an embodiment of the present invention, an embodiment of a network security entity classification method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] In this embodiment, a network security entity classification method is provided, which can be used for electronic devices such as computer devices. Figure 1 FIG. 1 is a flow chart of a network security entity classification method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0038] Step S101, obtaining entity data to be classified.

[0039] Specifically, the entity data to be classified may be entity data extracted from data obtained from various channels. The method of extracting entity data from data obtained from various channels may be the same as the conventional entity extraction method in the art, which will not be described in detail here.

[0040] Optionally, the data used can come from cybersecurity vocabulary books, the China National Vulnerability Database (CNVD), and entity corpora collected from the Internet.

[0041] Exemplarily, data obtained from various channels may be cleaned and structured to obtain entity data to be classified, or processed entity data to be classified may be obtained from other computer devices or databases.

[0042] Step S102: input the entity data to be classified into the fusion model to determine the fusion feature vector output by the fusion model.

[0043] Among them, the fusion model is a model that combines the BERT-large language model, multiple neural network models and attention mechanism in sequence.

[0044] Specifically, the output of the BERT-large language model is connected to the input of multiple neural network models, and the output of multiple neural network models is connected to the input of the attention mechanism. That is, the trained BERT-large language model, multiple trained neural network models, and the trained attention mechanism are spliced ​​to form a fusion model.

[0045] Exemplarily, the neural network model can be a convolutional neural network (CNN), a bidirectional long short-term memory network (BiLSTM), or a deep neural network (DNN).

[0046] BERT (Bidirectional Encoder Representations from Transformers) is a natural language processing technology based on the Transformer model. BERT provides efficient and accurate text representation for various NLP tasks through pre-training and fine-tuning. BERT-large is a pre-trained model obtained by further enlarging the model structure based on BERT and enhancing the model.

[0047] Attention Mechanism is a technology used to strengthen the neural network model's attention to important parts of the input data. The attention mechanism weights different parts of the input data, allowing the model to focus more on information related to the current task, thereby improving the model's entity classification accuracy.

[0048] Specifically, after obtaining the entity data to be classified, the entity data to be classified is input into the fusion model. The BERT-large language model in the fusion model first receives the entity data to be classified. The BERT-large language model can obtain the data representation of the information in the entity data to be classified (recorded as the first feature vector) through its own large language model advantage. Then, the first feature vector enters multiple neural network models, and the multiple neural network models extract text features respectively, and combine (concatenate) the extraction results to obtain the second feature vector. After that, the second feature vector enters the attention mechanism, and the attention mechanism extracts the important features of the second feature vector under the current entity classification task, and then obtains the fusion feature vector.

[0049] Step S103, inputting the fused feature vector into an activation function.

[0050] Step S104: determining whether the entity data to be classified is entity data in the field of network security based on the output value of the activation function.

[0051] Specifically, after determining the fused feature vector, the fused feature vector is input into the activation function for the final binary classification to determine the classification result, wherein the classification result includes whether the entity data to be classified is entity data in the field of network security and whether the entity data to be classified is not entity data in the field of network security.

[0052] Exemplarily, the activation function may be a softmax function, which may be as shown in formula (1):

[0053]

[0054] Where x represents a vector containing K real numbers, softmax(x) i Represents x i The output value of the vector can be regarded as mapping each element in the fused feature vector to the positive number space through an exponential function, and then normalizing all elements to obtain the probability distribution, that is, the classification result.

[0055] The fusion model constructed by the present invention has a significant improvement in accuracy and F1 value compared to the Text Convolutional Neural Network (TextCNN) in the test set of the present invention, which shows the effectiveness of the method of the present invention. In order to further verify the effectiveness of the method proposed by the present invention, the model constructed by the present invention still has an improvement compared to the classic deep learning method in the THUNews public data set, and the method of the present invention performs better in network security entity classification, with a greater improvement, indicating that the network security entity classification method based on the fusion model of BERT-large and the attention mechanism in the present invention has good field characteristics.

[0056] The network security entity classification method provided in this embodiment, after obtaining the entity data to be classified, inputs the entity data to be classified into a fusion model formed by a combination of a BERT-large language model, multiple neural network models and an attention mechanism to obtain a fusion feature vector, and then inputs the fusion feature vector into an activation function, and determines whether the entity data to be classified is entity data in the network security field based on the output value of the activation function, which can perform network security entity classification relatively quickly and accurately. This embodiment determines the classification result from entity features extracted from multiple dimensions, enhances the entity classification capability of the fusion model, improves the accuracy of the classification result, and also reduces the calculation dimension by focusing only on the current task features, thereby improving the classification efficiency.

[0057] In this embodiment, a network security entity classification method is provided, which can be used for electronic devices such as computer devices. Figure 2FIG. 1 is a flow chart of another network security entity classification method according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0058] Step S201, obtaining entity data to be classified.

[0059] For details, please see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0060] Step S202: construct a fusion model based on the initial BERT-large language model, multiple initial neural network models and the initial attention mechanism.

[0061] Exemplarily, the multiple neural network models in the fusion model include a deep pyramid convolutional neural network (DPCNN), a bidirectional long short-term memory network (BiLSTM), a first deep neural network, a second deep neural network, and a third deep neural network.

[0062] Specifically, Figure 3 As shown, the input end of the deep pyramid convolutional neural network (DPCNN), the input end of the bidirectional long short-term memory network (BiLSTM), and the input end of the first deep neural network (first DNN) are all connected to the output end of the BERT-large language model; the output end of the deep pyramid convolutional neural network, the output end of the bidirectional long short-term memory network, and the output end of the first deep neural network are all connected to the input end of the second deep neural network (second DNN). The output end of the second deep neural network is connected to the input end of the third deep neural network (third DNN), the output end of the third deep neural network is connected to the input end of the attention mechanism (Self-Attention), and the output end of the attention mechanism is connected to the activation function (soft-max).

[0063] The output of the activation function (soft-max) is yes (yes) and no (no), where yes means that the entity data is entity data in the field of network security, and no means that the entity data is not entity data in the field of network security.

[0064] Furthermore, if Figure 3 As shown, the fusion model also includes a batch normalization (Batch Normalization, BN) layer, which is arranged between the second deep neural network and the third deep neural network, that is, the input end of the batch normalization layer is connected to the output end of the second deep neural network, and the output end of the batch normalization layer is connected to the input end of the third deep neural network.

[0065] Specifically, the above step S202 at least includes steps S2021 to S2025:

[0066] Step S2021, creating a pre-training data set and a network security entity data set.

[0067] Specifically, the pre-training data set may be a set of entity data obtained by processing the acquired unstructured data, and the network security entity data set may be a set of entity data obtained by processing the acquired structured data or semi-structured data.

[0068] The pre-training dataset and the cybersecurity entity dataset are used to verify the effectiveness of the constructed model. The dataset construction process is to obtain a cybersecurity entity dictionary. The dictionary entities are manually screened through relevant books and cybersecurity vocabulary lists. The dictionary is used to expand entity data in Xlore, CN-Dbpedia, and Baidu Encyclopedia.

[0069] Exemplarily, the data used above comes from cybersecurity vocabulary books, CNVD, and network-collected entity corpora.

[0070] like Figure 4 As shown, after acquiring the data, the process of creating a pre-training data set can be specifically as follows: acquiring unstructured data, performing data cleaning, segmentation and sentence division, and automatic labeling through scripts to build a pre-training data set.

[0071] After acquiring the data, the process of creating a cybersecurity entity dataset can be as follows: clean the semi-structured data into structured data, for example, according to the DOM (Document Object Model) structure and tag attributes of the web page, use BeautifulSoup to locate and obtain specific tags and tag values ​​for the acquired HyperText Markup Language (HTML) data, and use Python scripts to perform Json parsing on the Application Programming Interface (API) data to form standardized structured data; then, manually label the acquired entity data, construct a cybersecurity entity classification dataset, and construct a training set, a validation set, and a test set in a ratio of 7:1:2.

[0072] DOM is a programming interface for representing and manipulating web documents. It presents the content of a web page (such as text, images, links, and other elements) in a tree structure, where each node represents an element in a web page (such as the element corresponding to an HTML tag), and the parent-child and sibling relationships between nodes correspond to the nesting and adjacency of web page elements in the page layout and structure. BeautifulSoup is the name of a Python library designed to make it easier to extract data from HTML and XML files.

[0073] Step S2022, obtain an initial BERT-large language model, multiple initial neural network models and an initial attention mechanism.

[0074] Specifically, the initial BERT-large language model may be a BERT-large language model that has not been trained with entity data in the field of network security, the initial neural network model may be a neural network model that has not been trained with entity data in the field of network security, and the initial attention mechanism may be an attention mechanism that has not been trained with entity data in the field of network security.

[0075] Step S2023: Train the initial BERT-large language model based on the pre-training data set to obtain an updated initial BERT-large language model.

[0076] Specifically, the language model in the present invention is not BERT-base (12 encoder layers, each layer has 768 hidden units, and the total number of parameters is 110M), but BERT-large (24 encoder layers, each layer has 1024 hidden units, and the total number of parameters is 340M) is used as the basic pre-training model.

[0077] After creating the pre-training dataset and the initial BERT-large language model, the model parameters of the initial BERT-large language model are retrained using the pre-training dataset, and the parameters of the BERT-large language model are fine-tuned so that the initial BERT-large language model can better understand, represent, and process network security-related corpora to enhance the overall classification effect of the model.

[0078] Exemplarily, two tasks, namely, predicting masked words (MSAK) (Masked Language Model, MLM) and predicting the next sentence (Next Sentence Prediction, NSP), are used to fine-tune the model parameters of the BERT-large language model.

[0079] Specifically, the BERT-large language model input consists of token embedding, segment embedding, and position embedding. That is, the input vector of the BERT-large language model can be expressed by the following formula (2):

[0080] v=v t +v s +v p (2)

[0081] Among them, v represents the input vector of the BERT-large language model. The BERT-large language model converts the received entity data into v for further processing. t represents the Token embedding vector, v s Represents the Segment embedding vector, v p Represents the Position embedding vector.

[0082] The three embedding vectors have the same dimensions and are obtained through embedding weight calculation and dimension conversion. The input vector v of the BERT-large language model is composed of the above three embedding vectors (v t 、v s and v p ) splicing representation.

[0083] Specifically, the initial BERT-large language model has an input length limit. Before the entity data in the pre-training dataset is input into the initial BERT-large language model, the entity data in the pre-training dataset needs to be pre-processed by sentence segmentation.

[0084] Step S2024, combining the updated initial BERT-large language model, multiple initial neural network models and the initial attention mechanism to form an initial fusion model.

[0085] Exemplarily, the fusion model may include a BERT-large language model, a DPCNN, a BiLSTM, a first DNN, a second DNN, a BN layer, a third DNN, and an attention mechanism. Accordingly, the initial fusion model may include an updated initial BERT-large language model, an initial DPCNN, an initial BiLSTM, an initial first DNN, an initial second DNN, an initial BN layer, an initial third DNN, and an initial attention mechanism.

[0086] Specifically, the input ends of the initial DPCNN, the initial BiLSTM and the initial first DNN are all connected to the output end of the initial BERT-large language model, the output ends of the initial DPCNN, the initial BiLSTM and the initial first DNN are all connected to the input end of the initial second DNN, the output end of the initial second DNN is connected to the input end of the initial BN layer, the output end of the initial BN layer is connected to the input end of the initial third DNN, and the output end of the initial third DNN is connected to the input end of the initial attention mechanism, thereby splicing to form an initial fusion model.

[0087] After the entity data passes through the BERT-large language model, the BERT-large language model can obtain the data representation of the entity text (recorded as the first feature vector), but the amount of representation data is large and further features are needed.

[0088] DPCNN extracts local significant features of the first feature vector. DPCNN can extract features through convolution and pooling. Specifically, firstly, a variety of convolution kernels of different sizes are used to extract features. The convolution calculation formula can be shown as formula (3). After convolution, maximum pooling is used to extract the most significant local features, and quickly reduce the feature dimension to speed up feature extraction.

[0089]

[0090] Among them, a i,j represents the result of a convolution, A and B represent the size of the convolution kernel, and w b is the current convolution kernel calculation bias, w m,n represents the weight value at the position (m, n) in the convolution kernel, x i+m,j+n Indicates the element value at the position corresponding to the convolution kernel in the input data. The size of the convolution kernel and the moving step will affect the data dimension of the convolution result. Different sizes of convolution kernels extract different features each time, and multiple convolutions are combined using the Ensemble idea.

[0091] BiLSTM extracts the time series features of the first feature vector, extracts the order information of the data and obtains the bidirectional information of the text. BiLSTM is a bidirectional LSTM. LSTM is mainly divided into three stages: forget, select memory and output. The forget stage corresponds to the forget gate. The input value of the previous state is judged whether it is forgotten after passing through the Sigmoid activation function layer. The select memory stage corresponds to the input gate. t After the Sigmoid activation function layer and the Tanh layer are processed, the important information is selected and memorized. The output stage corresponds to the output gate, and the addition result of the forget stage and the selected memory stage is subjected to the Tanh transformation, and the important information of the current state is selected and output after the input of the previous node is scaled.

[0092] The Sigmoid activation function can be expressed as formula (4):

[0093]

[0094] Among them, x s Indicates the value passed in by the previous state.

[0095] The first DNN performs weight calculation on each value in the first feature vector, extracts features layer by layer and reduces feature dimensions. The first DNN uses a ReLU (Rectified Linear Unit) activation function.

[0096] The ReLU activation function can be expressed as formula (5):

[0097] ReLu(x r )=max(0,x r ) (5)

[0098] Among them, x r Represents the output value of the first DNN. The ReLU activation function only transmits node information with weights greater than 0 and ignores node information with values ​​of zero, thereby reducing feature dimensions and computation.

[0099] The functions of the second and third DNNs are the same as those of the first DNN. The BN layer is used to improve the training effect of the DNN. By standardizing the input of each layer, the data is regularized into a uniform interval, internal covariate shift is reduced, the degree of data divergence is reduced, the learning difficulty of the network is reduced, and the training is accelerated and the stability of the model is improved.

[0100] During model training, for each mini-batch, the BN layer calculates its mean and standard deviation, and then normalizes it using the following formula (6):

[0101]

[0102] Among them, x is the input feature, is the mean of the mini-batch, is the variance of the mini-batch, and ∈ is a small constant used to prevent division by zero. Specifically, after normalization, the BN layer retains the distribution of the original data through formula (7).

[0103]

[0104] γ is the scaling factor and β is the offset factor.

[0105] After the first feature vector is extracted through multiple neural network models, all the extracted features are concatenated, and the features are scored through the attention mechanism (self-attention mechanism) to extract the important features of the current task.

[0106] In the self-attention mechanism, the query vector (Query), key vector (Key) and value vector (Value) come from the same source, that is, the query vector, key vector and value vector are obtained by linear transformation through the sequence information of the input text, and then multiplied by W respectively. Q , W K and W V .W Q , W K and W V is the weight matrix that the model needs to train, and the calculation formula (8) of the self-attention mechanism can be:

[0107]

[0108] Among them, Q represents Query, K represents Key, V represents Value, and the similarity between Q and K is calculated using the dot product method. represents the scaling factor, scaling the attention value, d k Represents the dimension of the key vector; the similarity is normalized by softmax to obtain the attention weight, and the attention weight is weighted and summed with the corresponding V to obtain the final attention representation for the final classification.

[0109] For example, the loss function of the attention mechanism may be a cross-entropy loss function, and the cross-entropy loss function may be as shown in formula (9):

[0110] Loss=-(ylog(p)+(1-y)log(1-p)) (9)

[0111] Among them, y is the true label, which indicates the true category of the sample, and its value is 0 or 1; p is the model's predicted probability that the sample belongs to the positive class, and its value range is between 0 and 1.

[0112] The cross entropy loss function can better transform the training objective of the model into minimizing the loss in the binary classification task, calculate the gradient through the back propagation algorithm, and use gradient descent to update the model parameters, so that the loss function gradually decreases and the performance of the model is improved.

[0113] Step S2025: training the initial fusion model based on the network security entity data set to obtain a fusion model.

[0114] Specifically, after the initial fusion model is obtained, the network security entity data set is used to train the initial fusion model, and the trained initial fusion model is the fusion model.

[0115] Exemplarily, during the training process, the entity data first passes through the updated initial BERT-large language model. After the updated initial BERT-large language model obtains the data representation of the entity text (recorded as the first feature vector), the first feature vector is respectively subjected to feature extraction by multiple neural network models including the initial DPCNN, the initial BiLSTM and the initial first DNN. After the extracted multi-model features are concatenated, the initial second DNN, the initial BN layer and the initial third DNN are used to further perform feature extraction and dimensionality reduction to obtain the second feature vector.

[0116] Then, the second feature vector enters the initial attention mechanism, scores the features through the self-attention mechanism, extracts the important features of the current task, and obtains the fused feature vector. After that, the fused feature vector is input into the activation function to determine the classification result, the loss is determined based on the loss function, and the parameters of the updated initial BERT-large language model, initial DPCNN, initial BiLSTM, initial first DNN, initial second DNN, initial BN layer, initial third DNN and initial attention mechanism are adjusted based on the loss, and then the above process is repeated until the loss is below the preset loss, and the trained initial fusion model is obtained.

[0117] Specifically, different parts of the fusion model constructed in the present invention use different learning rates, and the learning rate used by the BERT-large language model part is different from the learning rates used by multiple neural network models and attention mechanisms.

[0118] In one example, the learning rate used by the BERT-large language model part can be 1e-5, and the parameters of the BERT-large language model are fine-tuned for the second time. Using a relatively low learning rate can retain the good generalization of the large language model to handle new entity content. After tuning, the learning rate used by multiple neural network models, BN layers, and attention mechanisms can be 1e-4.

[0119] Specifically, the four hyperparameters of learning rate, batch size, number of iterations, and Dropout can be selected using the grid search method. The activation functions use ReLu, Tanh, and Sigmoid functions, and different activation functions are used in different network layers.

[0120] Step S203: input the entity data to be classified into the fusion model to determine the fusion feature vector output by the fusion model.

[0121] For details, please see Figure 1Step S102 of the illustrated embodiment will not be described in detail here.

[0122] Step S204, inputting the fused feature vector into an activation function.

[0123] Step S205: determining whether the entity data to be classified is entity data in the field of network security based on the output value of the activation function.

[0124] For details, please see Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0125] The network security entity classification method provided in this embodiment determines the classification results from entity features extracted from multiple dimensions, enhances the entity classification capability of the fusion model, improves the accuracy of the classification results, and reduces the calculation dimension by focusing only on the current task features, thereby improving the classification efficiency. Moreover, by fine-tuning the BERT-large language model twice, the model's ability to represent network security-related entity information can be enhanced, and different entities can be better represented and distinguished.

[0126] This embodiment also provides a network security knowledge graph construction method, which can be used in electronic devices such as computers. Figure 5 is a flow chart of a method for constructing a network security knowledge graph according to an embodiment of the present invention. Figure 5 As shown, the method comprises the following steps:

[0127] Step S501, obtaining entity data to be classified.

[0128] For details, please see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0129] Step S502: input the entity data to be classified into the fusion model, and determine the fusion feature vector output by the fusion model.

[0130] For details, please see Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0131] Step S503: input the fused feature vector into an activation function.

[0132] Step S504: determining whether the entity data to be classified is entity data in the field of network security based on the output value of the activation function.

[0133] For details, please see Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0134] Step S505, when the entity data to be classified is entity data in the field of network security, the entity data to be classified is added to the network security knowledge graph entity library.

[0135] Specifically, the network security entity method provided by the present invention can be applied to the construction of network security knowledge graphs, such as Figure 6 As shown, after acquiring new entity data (i.e., entity data to be classified) in the network, it is determined whether the new entity data is entity data in the field of network security. When it is determined that the entity data to be classified is not entity data in the field of network security, the entity data to be classified is discarded; when it is determined that the entity data to be classified is entity data in the field of network security, the entity data to be classified is added to the network security knowledge graph entity library to update the network security knowledge graph entity library and ensure the domain relevance of the constructed knowledge graph.

[0136] In this embodiment, a network security entity classification device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0137] This embodiment provides a network security entity classification device, such as Figure 7 As shown, including:

[0138] An acquisition module 701 is used to acquire entity data to be classified;

[0139] A first determination module 702 is used to input the entity data to be classified into a fusion model, and determine a fusion feature vector output by the fusion model, wherein the fusion model is a model in which a BERT-large language model, a plurality of neural network models, and an attention mechanism are sequentially combined;

[0140] An input module 703 is used to input the fused feature vector into an activation function;

[0141] The second determination module 704 is used to determine whether the entity data to be classified is entity data in the network security field based on the output value of the activation function.

[0142] In some optional embodiments, the multiple neural network models include a deep pyramid convolutional neural network, a bidirectional long short-term memory network, a first deep neural network, a second deep neural network and a third deep neural network; the input end of the deep pyramid convolutional neural network, the input end of the bidirectional long short-term memory network and the input end of the first deep neural network are all connected to the output end of the BERT-large language model; the output end of the deep pyramid convolutional neural network, the output end of the bidirectional long short-term memory network and the output end of the first deep neural network are all connected to the input end of the second deep neural network; the output end of the second deep neural network is connected to the input end of the third deep neural network, and the output end of the third deep neural network is connected to the input end of the attention mechanism.

[0143] In some optional embodiments, the fusion model further includes a batch normalization layer, an input end of the batch normalization layer is connected to an output end of the second deep neural network, and an output end of the batch normalization layer is connected to an input end of the third deep neural network.

[0144] In some optional implementations, the activation function is:

[0145]

[0146] Where x represents a vector containing K real numbers, softmax(x) i Represents x i Vector of output values.

[0147] In some optional embodiments, the device further comprises:

[0148] A building module for building a fusion model based on the initial BERT-large language model, multiple initial neural network models, and the initial attention mechanism.

[0149] In some optional embodiments, the building blocks include:

[0150] A first creation unit, used to create a pre-training data set and a network security entity data set;

[0151] An acquisition unit, used to acquire an initial BERT-large language model, multiple initial neural network models, and an initial attention mechanism;

[0152] A first training unit is used to train the initial BERT-large language model based on the pre-training data set to obtain an updated initial BERT-large language model;

[0153] The second creation unit is used to combine the updated initial BERT-large language model, multiple initial neural network models and the initial attention mechanism to form an initial fusion model;

[0154] The second training unit is used to train the initial fusion model based on the network security entity data set to obtain the fusion model.

[0155] This embodiment provides a network security knowledge graph construction device, such as Figure 8 As shown, including:

[0156] An acquisition module 701 is used to acquire entity data to be classified;

[0157] A first determination module 702 is used to input the entity data to be classified into a fusion model, and determine a fusion feature vector output by the fusion model, wherein the fusion model is a model in which a BERT-large language model, a plurality of neural network models, and an attention mechanism are sequentially combined;

[0158] An input module 703 is used to input the fused feature vector into an activation function;

[0159] A second determination module 704, configured to determine whether the entity data to be classified is entity data in the field of network security based on the output value of the activation function;

[0160] The adding module 801 is used to add the entity data to be classified into the network security knowledge graph entity library when the entity data to be classified is entity data in the network security field.

[0161] Specifically, the network security knowledge graph construction device includes the network security entity classification device provided by any of the above embodiments.

[0162] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0163] The network security entity classification device and network security knowledge graph construction device in this embodiment are presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0164] The embodiment of the present invention also provides a computer device, such as Fig. 9As shown, the computer device includes: one or more processors 910, memory 920, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 910 is taken as an example.

[0165] The processor 910 may be a central processing unit, a network processor or a combination thereof. The processor 910 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0166] The memory 920 stores instructions executable by at least one processor 910, so that the at least one processor 910 executes the method shown in the above embodiment.

[0167] The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 920 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 920 may optionally include a memory remotely arranged relative to the processor 910, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0168] The memory 920 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 920 may also include a combination of the above types of memory.

[0169] The computer device also includes a communication interface 930, which is used for the computer device to communicate with other devices or a communication network.

[0170] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0171] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0172] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0173] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0174] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0175] In the description of this specification, the description with reference to the terms "this embodiment", "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0176] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0177] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the present invention.

Claims

1. A network security entity classification method, characterized in that: The method comprises: Obtain entity data to be classified; Inputting the entity data to be classified into the fusion model, and determining the fusion feature vector output by the fusion model, wherein the fusion model is a model that sequentially combines a BERT-large language model, multiple neural network models, and an attention mechanism; Inputting the fused feature vector into an activation function; Based on the output value of the activation function, it is determined whether the entity data to be classified is entity data in the field of network security.

2. The method according to claim 1, characterized in that The multiple neural network models include a deep pyramid convolutional neural network, a bidirectional long short-term memory network, a first deep neural network, a second deep neural network, and a third deep neural network; The input end of the deep pyramid convolutional neural network, the input end of the bidirectional long short-term memory network, and the input end of the first deep neural network are all connected to the output end of the BERT-large language model; The output end of the deep pyramid convolutional neural network, the output end of the bidirectional long short-term memory network and the output end of the first deep neural network are all connected to the input end of the second deep neural network; The output end of the second deep neural network is connected to the input end of the third deep neural network, and the output end of the third deep neural network is connected to the input end of the attention mechanism.

3. The method according to claim 2, characterized in that The fusion model also includes a batch normalization layer, an input end of the batch normalization layer is connected to an output end of the second deep neural network, and an output end of the batch normalization layer is connected to an input end of the third deep neural network.

4. The method according to any one of claims 1 to 3, characterized in that The activation function is: Where x represents a vector containing K real numbers, softmax(x) i Represents x i Vector of output values.

5. The method according to any one of claims 1 to 3, characterized in that Before inputting the entity data to be classified into the fusion model and determining the fusion feature vector output by the fusion model, the method further includes: The fusion model is constructed based on an initial BERT-large language model, multiple initial neural network models, and an initial attention mechanism.

6. The method according to claim 5, characterized in that The fusion model is constructed based on the initial BERT-large language model, multiple initial neural network models and the initial attention mechanism, including: Create pre-training datasets and cybersecurity entity datasets; Obtain the initial BERT-large language model, multiple initial neural network models, and initial attention mechanism; Based on the pre-training data set, the initial BERT-large language model is trained to obtain an updated initial BERT-large language model; Combining the updated initial BERT-large language model, the multiple initial neural network models, and the initial attention mechanism to form an initial fusion model; The initial fusion model is trained based on the network security entity data set to obtain the fusion model.

7. A method for constructing a network security knowledge graph, characterized in that: The method comprises: Obtain entity data to be classified; Inputting the entity data to be classified into the fusion model, and determining the fusion feature vector output by the fusion model, wherein the fusion model is a model that sequentially combines a BERT-large language model, multiple neural network models, and an attention mechanism; Inputting the fused feature vector into an activation function; Based on the output value of the activation function, determining whether the entity data to be classified is entity data in the field of network security; When the entity data to be classified is entity data in the field of network security, the entity data to be classified is added to the network security knowledge graph entity library.

8. A network security entity classification device, characterized in that: The device comprises: An acquisition module, used to acquire entity data to be classified; A first determination module is used to input the entity data to be classified into a fusion model, and determine a fusion feature vector output by the fusion model, wherein the fusion model is a model in which a BERT-large language model, a plurality of neural network models, and an attention mechanism are sequentially combined; An input module, used for inputting the fused feature vector into an activation function; The second determination module is used to determine whether the entity data to be classified is entity data in the field of network security based on the output value of the activation function.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the network security entity classification method described in any one of claims 1 to 6, or executes the network security knowledge graph construction method described in claim 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the network security entity classification method described in any one of claims 1 to 6, or to execute the network security knowledge graph construction method described in claim 7.

Images