Missing information complementing method and device based on advanced semantic embedding and graph embedding

Abstract

The invention relates to a missing information complementing method and device based on advanced semantic embedding and graph embedding. The method comprises the following steps: acquiring network security intelligence data and a corresponding network security intelligence knowledge graph; after preprocessing the data, converting each entity text in the data into a numerical vector meeting a preset dimension by utilizing a preset Word2Vec model; calculating the weight of each edge in the network security intelligence knowledge graph, and constructing a random walk sequence based on the weights to obtain vectorization representation of nodes of the network security intelligence knowledge graph; and obtaining a semantic embedding completion result according to the numerical vector, obtaining a graph embedding completion result according to vectorization representation of nodes of the network security intelligence knowledge graph, obtaining a final missing information completion result according to the two completion results, and completing the information. Therefore, the reliability of a final completion result is improved by vectorizing information data and information in the knowledge graph and integrating semantic embedding and graph embedding model completion results.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to a method and device for completing missing information based on high-level semantic embedding and graph embedding. Background Art

[0002] The effectiveness and timeliness of cybersecurity intelligence are the key to building a cybersecurity intelligence system. In the process of building a cybersecurity intelligence knowledge graph, information loss often occurs, mainly reflected in the loss of entity attribute values, which not only affects the integrity of entity data in the knowledge graph, but also affects the data quality obtained by information query. For example, the lack of key information such as vulnerability risk level or attack method may make it impossible for security personnel to judge the extent of the damage caused by the attack and the mitigation measures that can be taken in the first time.

[0003] In the related technology, the knowledge graph is completed by segmenting the text information of complete triples and the text information of missing triples, obtaining the relationship between them, and using the feature vector obtained by the language representation model.

[0004] However, although the above-mentioned technical means can complete the missing information of the knowledge graph, they lack semantic information and vectorized representation of graph embedded nodes. When constructing the cybersecurity intelligence knowledge graph, the description of entities and relationships is not accurate enough. At the same time, it is difficult to handle complex information, which needs to be solved urgently. Summary of the invention

[0005] The present invention provides a method and device for completing missing information based on advanced semantic embedding and graph embedding, so as to solve the problems in related technologies such as lack of semantic information and vectorized representation of graph embedding nodes, inaccurate description of entities and relationships when constructing a network security intelligence knowledge graph, and inability to process complex information. The method complements the missing information of the network security intelligence knowledge graph, and directly processes and analyzes the complex information in the knowledge graph by a computer, thereby improving computing efficiency and accuracy.

[0006] To achieve the above objectives, a first aspect of the present invention proposes a missing information completion method based on high-level semantic embedding and graph embedding, comprising the following steps:

[0007] Acquire network security intelligence data and a network security intelligence knowledge graph corresponding to the network security intelligence data;

[0008] Preprocessing the network security intelligence data, and using a pre-trained Word2Vec model to convert each entity text in the preprocessed network security intelligence data into a numerical vector that meets a preset dimension;

[0009] Calculate the weight of each edge in the network security intelligence knowledge graph, and construct a random walk sequence based on the weight of each edge, and obtain a vectorized representation of the network security intelligence knowledge graph node according to the random walk sequence;

[0010] A semantic embedding completion result is obtained according to the numerical vector, and a graph embedding completion result is obtained according to the vectorized representation of the nodes of the network security intelligence knowledge graph, and a final missing information completion result is obtained according to the semantic embedding completion result and the graph embedding completion result, so as to perform information completion according to the final missing information completion result.

[0011] According to an embodiment of the present invention, the step of calculating the weight of each edge in the network security intelligence knowledge graph and constructing a random walk sequence based on the weight of each edge includes:

[0012] Determine an importance adjustment factor of each edge and a node degree adjustment factor of each node in the network security intelligence knowledge graph, and calculate a weight of each edge in the network security intelligence knowledge graph according to the importance adjustment factor, the node degree adjustment factor and a preset basic weight;

[0013] Based on the weight of each edge in the network security intelligence knowledge graph and a preset walking strategy, a random walk is performed in the network security intelligence knowledge graph to generate a random walk sequence of the network security intelligence knowledge graph.

[0014] According to an embodiment of the present invention, obtaining a vectorized representation of the network security intelligence knowledge graph node according to the random walk sequence includes:

[0015] Initializing a vector for each node in the network security intelligence knowledge graph;

[0016] Based on a preset loss function and a context window, determining positive sample nodes and negative sample nodes according to the random walk sequence;

[0017] The preset model is trained using the positive sample nodes, the vectors corresponding to the positive sample nodes, the negative sample nodes and the vectors corresponding to the negative sample nodes to obtain a vectorized model for vectorizing the nodes, so as to obtain the vectorized representation of the network security intelligence knowledge graph nodes through the vectorized model.

[0018] According to one embodiment of the present invention, the weight of each edge in the network security intelligence knowledge graph is:

[0019] w(u,v)=w relation_type ×degree_adj(v)×importance_adj(u,v);

[0020] Among them, w(u,v) is the weight of each edge in the network security intelligence knowledge graph, w relation_type It is the basic weight setting for the relationship between different intelligence types, degree_adj(v) is the degree adjustment factor of each node, and importance_adj(u,v) is the importance adjustment factor.

[0021] According to an embodiment of the present invention, the preset roaming strategy is:

[0022]

[0023] Among them, P(c i =x|c i-1 =v,c i-2 =t) is the transition probability, α pq (t, x) is the relative position adjustment factor between the node t visited in the previous step and the neighbor node x currently considered, and Weight vx is the weight of edge (v, x), Σ n∈N(v) α pq (t, n) is the sum of all v's neighbor nodes, Weight vn is the weight of the edge corresponding to all v's neighbor nodes.

[0024] According to the missing information completion method based on advanced semantic embedding and graph embedding proposed in the embodiment of the present invention, the obtained network security intelligence data is preprocessed, and each entity text in the preprocessed network security intelligence data is converted into a numerical vector using a pre-trained Word2Vec model; then the weight of each edge in the corresponding network security intelligence knowledge graph is calculated, and a random walk sequence is constructed based on the weight of each edge to obtain a vectorized display of the network security intelligence knowledge graph node; finally, according to the numerical vector and the vectorized display of the network security intelligence knowledge graph node, the semantic embedding completion result and the graph embedding completion result are calculated, and the final missing information completion result is calculated according to the two completion results, and the missing information is completed. Thus, by vectorizing the information in the intelligence data and the knowledge graph, and combining the completion results of the semantic embedding and graph embedding models to improve the reliability of the final completion result, the completion of the missing information of the network security intelligence knowledge graph is achieved, and the auxiliary computing tool can be used to directly process and analyze the complex information in the knowledge graph, thereby improving the computing efficiency and accuracy.

[0025] To achieve the above-mentioned purpose, a second embodiment of the present invention proposes a missing information completion device based on high-level semantic embedding and graph embedding, comprising:

[0026] An acquisition module, used to acquire network security intelligence data and a network security intelligence knowledge graph corresponding to the network security intelligence data;

[0027] A preprocessing module, used to preprocess the network security intelligence data, and use a pre-trained Word2Vec model to convert each entity text in the preprocessed network security intelligence data into a numerical vector that meets a preset dimension;

[0028] A walk module, used to calculate the weight of each edge in the network security intelligence knowledge graph, and to construct a random walk sequence based on the weight of each edge, and to obtain a vectorized representation of the network security intelligence knowledge graph node according to the random walk sequence;

[0029] An information completion module is used to obtain a semantic embedding completion result based on the numerical vector, and to obtain a graph embedding completion result based on the vectorized representation of the network security intelligence knowledge graph node, and to obtain a final missing information completion result based on the semantic embedding completion result and the graph embedding completion result, so as to complete the information according to the final missing information completion result.

[0030] According to one embodiment of the present invention, the walking module is specifically used for:

[0031] Determine an importance adjustment factor of each edge and a node degree adjustment factor of each node in the network security intelligence knowledge graph, and calculate a weight of each edge in the network security intelligence knowledge graph according to the importance adjustment factor, the node degree adjustment factor and a preset basic weight;

[0032] Based on the weight of each edge in the network security intelligence knowledge graph and a preset walking strategy, a random walk is performed in the network security intelligence knowledge graph to generate a random walk sequence of the network security intelligence knowledge graph.

[0033] According to one embodiment of the present invention, the walking module is specifically used for:

[0034] Initializing a vector for each node in the network security intelligence knowledge graph;

[0035] Based on a preset loss function and a context window, determining positive sample nodes and negative sample nodes according to the random walk sequence;

[0036] The preset model is trained using the positive sample nodes, the vectors corresponding to the positive sample nodes, the negative sample nodes and the vectors corresponding to the negative sample nodes to obtain a vectorized model for vectorizing the nodes, so as to obtain the vectorized representation of the network security intelligence knowledge graph nodes through the vectorized model.

[0037] According to one embodiment of the present invention, the weight of each edge in the network security intelligence knowledge graph is:

[0038] w(u,v)=w relation_type ×degree_adj(v)×importance_adj(u,v);

[0039] Among them, w(u,v) is the weight of each edge in the network security intelligence knowledge graph, w relation_type It is the basic weight setting for the relationship between different intelligence types, degree_adj(v) is the degree adjustment factor of each node, and importance_adj(u,v) is the importance adjustment factor.

[0040] According to an embodiment of the present invention, the preset roaming strategy is:

[0041]

[0042] Among them, P(c i =x|c i-1 =v,c i-2 =t) is the transition probability, α pq (t, x) is the relative position adjustment factor between the node t visited in the previous step and the neighbor node x currently considered, and Weight vx is the weight of edge (v, x), Σ n∈N(v) α pq (t, n) is the sum of all v's neighbor nodes, Weight vn is the weight of the edge corresponding to all v's neighbor nodes.

[0043] According to the missing information completion device based on advanced semantic embedding and graph embedding proposed in the embodiment of the present invention, the acquired network security intelligence data is preprocessed, and each entity text in the preprocessed network security intelligence data is converted into a numerical vector using a pre-trained Word2Vec model; then the weight of each edge in the acquired corresponding network security intelligence knowledge graph is calculated, and a random walk sequence is constructed based on the weight of each edge to obtain a vectorized display of the network security intelligence knowledge graph node; finally, according to the numerical vector and the vectorized display of the network security intelligence knowledge graph node, the semantic embedding completion result and the graph embedding completion result are calculated, and the final missing information completion result is calculated according to the two completion results, and the missing information is completed. Thus, by vectorizing the information in the intelligence data and the knowledge graph, and combining the completion results of the semantic embedding and graph embedding models to improve the reliability of the final completion result, the completion of the missing information of the network security intelligence knowledge graph is achieved, and the auxiliary computing tool can be used to directly process and analyze the complex information in the knowledge graph, thereby improving the computing efficiency and accuracy.

[0044] To achieve the above-mentioned objectives, the third aspect of the present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the missing information completion method based on high-level semantic embedding and graph embedding as described in the above-mentioned embodiments.

[0045] To achieve the above objectives, the fourth aspect of the present invention proposes a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the missing information completion method based on high-level semantic embedding and graph embedding as described in the above embodiments.

[0046] To achieve the above objectives, the fifth aspect of the present invention proposes a computer program product, including a computer program, which, when executed by a processor, is used to implement the missing information completion method based on high-level semantic embedding and graph embedding as described in the above embodiments.

[0047] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0049] Figure 1 A flowchart of a missing information completion method based on high-level semantic embedding and graph embedding provided according to an embodiment of the present invention;

[0050] Figure 2 A schematic table of common English stop words provided according to a specific embodiment of the present invention;

[0051] Figure 3 A schematic diagram of the change of the original text of the network security intelligence data after preprocessing by removing stop words according to a specific embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the change of the knowledge graph connection structure before and after the introduction of super nodes according to a specific embodiment of the present invention;

[0053] Figure 5 A schematic diagram of a random walk sequence window provided according to a specific embodiment of the present invention;

[0054] Figure 6 A schematic diagram of the principle of a missing information completion algorithm provided according to a specific embodiment of the present invention;

[0055] Figure 7This is a block diagram of an example of a missing information completion device based on advanced semantic embedding and graph embedding of the present application;

[0056] Figure 8 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention.

[0057] Reference numerals:

[0058] 10-missing information completion device based on high-level semantic embedding and graph embedding, 100-acquisition module, 200-preprocessing module, 300-wandering module, 400-information completion module, 801-memory, 802-processor, 803-communication interface. DETAILED DESCRIPTION

[0059] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0060] A missing information completion method and device based on advanced semantic embedding and graph embedding according to an embodiment of the present invention will be described below with reference to the accompanying drawings. First, a missing information completion method based on advanced semantic embedding and graph embedding according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0061] Figure 1 It is a flowchart of a missing information completion method based on high-level semantic embedding and graph embedding provided by an embodiment of the present invention.

[0062] like Figure 1 As shown in FIG. 1 , the missing information completion method based on high-level semantic embedding and graph embedding includes the following steps:

[0063] In step S101, network security intelligence data and a network security intelligence knowledge graph corresponding to the network security intelligence data are obtained.

[0064] Specifically, the embodiments of the present invention can crawl data contents of different URLs through crawler technology, and store the crawled data contents and sources in an index table, extract entities and relationships from the data contents, and obtain network security intelligence data; then introduce super nodes to eliminate redundant entities and relationships in the network security intelligence data, so as to construct a network security intelligence knowledge graph corresponding to the network security intelligence data.

[0065] In step S102, the network security intelligence data is preprocessed, and each entity text in the preprocessed network security intelligence data is converted into a numerical vector that meets a preset dimension using a pre-trained Word2Vec model.

[0066] It should be noted that preprocessing cybersecurity intelligence data is the key to converting each entity text in cybersecurity intelligence data into a numerical vector. Its purpose is to prepare and optimize text data to ensure that the vectorization process can efficiently and accurately capture the semantic information of the text. The preprocessing process includes a series of standardization and cleaning operations such as removing irrelevant characters, unifying uppercase and lowercase, word segmentation, removing stop words, and optional stemming. These steps help reduce data noise and vocabulary diversity, laying the foundation for the subsequent training of semantic embedding models.

[0067] Specifically, the preprocessing process includes a series of standardization and cleaning operations such as removing irrelevant characters, unifying uppercase and lowercase characters, word segmentation, removing stop words, and optional stem extraction. These steps help reduce data noise and vocabulary diversity, laying the foundation for the subsequent training of semantic embedding models. An advanced semantic information preprocessing algorithm provided by an embodiment of the present invention is as follows:

[0068] Input: English text dataset Texts

[0069] Output: preprocessed text dataset ProcessedTexts

[0070] 1: Initialize an empty list ProcessedTexts

[0071] 2: Load the English stop word list stop_words

[0072] 3: for text∈Texts do

[0073] 4: text_lowercase←tolower(text)

[0074] 5: text_cleaned←remove_special(text_lowercase)

[0075] 6: words←tokenize(text_cleaned)

[0076] 7: words_filtered←filter_stopwords(words, stop_words)

[0077] 8: processed_text←combine(words_filtered)

[0078] 9: Add processed_text to ProcessedTexts

[0079] 10: end for

[0080] 11: return ProcessedTexts

[0081] Among them, the core steps of preprocessing include: unification of uppercase and lowercase letters, data cleaning and text standardization. First, unification of uppercase and lowercase letters is performed. Considering that the difference in uppercase and lowercase letters in English text will cause the same word to be regarded as different terms, converting all texts to lowercase letters can reduce the number of word variants, thereby simplifying the subsequent text processing process. Secondly, in the data cleaning stage, the present invention focuses on removing non-core elements that may interfere with text analysis, including special characters, extra spaces or line breaks, etc., while retaining tags that are of great significance to intelligence description, in preparation for subsequent further processing. Finally, text standardization. Text standardization is further refined into two steps: word segmentation and stop word removal. Among them, word segmentation processing is to split the text into the smallest meaningful unit, that is, a word or phrase. The embodiment of the present invention can realize word segmentation processing by means of some open source natural language processing tools, for example, using the open source NLTK (Natural Language Toolkit) library. Stop word removal refers to removing words that appear frequently in the text but contribute less to understanding the text theme, such as common English words such as "the", "is", "at", etc., to reduce data dimensions and improve the efficiency and accuracy of subsequent processing.

[0082] For example, Figure 2 As shown, Figure 2 This is a schematic table of common English stop words provided according to a specific embodiment of the present invention. The table stores some common English stop words that frequently appear in texts but contribute little to understanding the text topic. The original text of network security intelligence data after preprocessing by removing stop words is as follows: Figure 3 After completing the preprocessing of the network security intelligence data, each entity text in the preprocessed network security intelligence data needs to be converted into a numerical vector, which is achieved by the pre-trained Word2Vec model in the embodiment of the present invention.

[0083] It should be noted that the Word2Vec model generates word vectors by learning the contextual relationship of words in the text. During the training process, Word2Vec has two main architectures: Continuous Bag of Words (CBOW) and Skip-Gram. The core idea of ​​the CBOW model is to predict the target word in a given context vocabulary, while the Skip-Gram model is just the opposite. It takes the target vocabulary as input and predicts the words in its context environment through the current word. This method is particularly suitable for capturing the representation of rare words and maintaining the similarity of word meanings in the word embedding space. Considering that the Skip-Gram model performs better when processing small amounts of data and rare words, and relatively rare words may appear in network security intelligence, therefore, for the vectorized representation of high-level semantic information of network security intelligence, the embodiment of the present invention uses the Skip-Gram model.

[0084] In order to enable those skilled in the art to further understand the Word2Vec model trained in the embodiment of the present invention, the training process of the Word2Vec model will be described in detail below.

[0085] Specifically, the goal of the Skip-Gram model is to predict the context (surrounding words) given a target word (central word) by maximizing the log-likelihood function of the probability of the context words appearing when the target word is given. This process can be formally expressed as the following objective function:

[0086]

[0087] Among them, w t is the target word, w t+j is the context vocabulary, c is the context window size, and T is the total number of words in the text.

[0088] The core of the above objective function is to calculate the given target word W I When the context word W O The conditional probability of occurrence. This probability is calculated by the following softmax function:

[0089]

[0090] Among them, v WI is the target word W I The vector representation of v' WO is the vector representation of the context word Wo, |V| is the size of the vocabulary. The numerator exp(v' WO T v WI) measures the similarity between the target word and the context word. The denominator is the exponential sum of the similarities of all words in the entire vocabulary as context words, ensuring the normalization of the conditional probability.

[0091] According to formula (2), for the softmax function, its denominator needs to traverse each word W in the vocabulary, calculate the exponential function once for each word, and then sum it up. For a vocabulary with a large number of words, each probability calculation may require millions of exponential calculations and summations, which is not feasible in practice, especially when the model needs to be trained repeatedly.

[0092] In order to solve the problem of huge amount of calculation of softmax function, the present invention adopts negative sampling technology in the actual training model process. Negative sampling approximates the calculation of the denominator by selecting a small number of negative samples (i.e., words that are not in the context of the current target word). In this way, only the vectors of the target word and a few negative samples need to be updated each time, instead of every word in the vocabulary, which greatly reduces the amount of calculation.

[0093] In negative sampling, the invention is the target word Wo and each word W in its context i Positive and negative examples are defined:

[0094] A positive example indicates that there is a positive relationship between the context word and the target word:

[0095]

[0096] Negative examples indicate that there is no contextual relationship between the selected negative sample words and the target words:

[0097]

[0098] In the above formula, x WO is the vector representation of the target word Wo, θ Wi is with the word W i The associated parameters, σ is the sigmiod function, and neg is the number of negative samples.

[0099] The goal of this invention is to maximize the joint probability of positive and negative examples, which can be achieved by minimizing the following likelihood function:

[0100]

[0101] In the above formula (5), y i is an indicator variable, which is 1 for the context word of the target word and 0 for the negative sample. The model parameters θ can be updated by gradient descent on the likelihood function L Wi , to better predict the context words of the target word Wo.

[0102] In addition, considering that in cybersecurity intelligence text materials, some high-frequency words appear far more frequently than most professional terms or low-frequency words. If negative sampling is performed directly according to the original word frequency, these high-frequency words will be oversampled, which will have limited improvement on the semantic capture and generalization capabilities of the model. In order to effectively utilize the corpus and improve the efficiency and quality of model training, the present invention adjusts the probability of each word being selected as a negative sample:

[0103]

[0104] Among them, f(W i ) represents the word W i The frequency in the entire dataset, the denominator is the sum of the 3 / 4 power of all word frequencies, used to normalize the probability. The choice of 3 / 4 as the exponent is based on experimental observations to find a more appropriate value.

[0105] For example, taking the information of "Jet Database Engine Remote Code Execution Vulnerability" as an example, there is: "A remotecode execution vulnerability exists when the Windows Jet Database Engine improperly handles objects in memory, aka'Jet Database Engine Remote CodeExecution Vulnerability'". With "vulnerability" as the target word and a context window of 2, there may be the following training samples:

[0106] Target word: “vulnerability”

[0107] Context (positive sample): "code", "execution", "exists", "when"

[0108] Assume that the negative samples (randomly selected non-context words) are: "Windows", "Jet".

[0109] In the Skip-Gram model, for each positive sample, the model tries to adjust the parameters so that v' WO and W I The inner product of the words is as large as possible (increasing the probability of these words appearing together), and for each negative sample, the model will try to adjust the parameters to make their inner product as small as possible (reducing the probability of these words appearing together).

[0110] Through such a training process, the model can eventually learn vector representations that reflect the complex semantic relationships between words. For example, through training, the vectors of words related to security vulnerabilities such as "vulnerability" and "code" and "execution" will be close to each other in the vector space, while the vectors of irrelevant words such as "Windows" and "Jet" (if they are selected as negative samples) will be farther away.

[0111] Referring to the training process of the Word2Vec model mentioned above, the overall process algorithm of Word2Vec model training is as follows:

[0112] Input: text dataset D (preprocessed entity description set)

[0113] Output: trained high-level semantic vectorization model M

[0114] l: Initialize the model parameters, including word vectors and context vectors

[0115] 2: Choose the appropriate context window size c and negative sampling number N

[0116] 3: Iterate for each text in dataset D

[0117] 4: Segment the text to get the word sequence

[0118] 5: for each target word do

[0119] 6: According to the context window size c, identify its context word set

[0120] 7: Randomly generate N negative samples and determine the positive and negative sample sets for each target word

[0121] 8: Using the Skip-Gram model and the sample set obtained by negative sampling, calculate the gradient to maximize the log likelihood of positive samples and minimize the log likelihood of negative samples

[0122] 9: Update the parameters of the word vector and context vector by stochastic gradient descent to optimize the overall loss

[0123] Loss Function

[0124] 10: end for

[0125] 1l: end for

[0126] 12: Repeat the above steps until the model converges or reaches the preset number of iterations

[0127] 13: return the trained model M

[0128] After completing the training of the model, the present invention needs to represent the high-level semantic information of each entity in the network security intelligence as a numerical vector for subsequent analysis and processing.

[0129] The main process is as follows: For each text entity, initialize a zero vector whose dimension is the same as the word vector dimension generated by the model. Traverse each word in the entity text. If the word is in the vocabulary of the model, add the vector of the word to the initialized zero vector. At the same time, record the number of word vectors added. Divide the accumulated word vector by the number of words to obtain the average word vector of the entity text as the vector representation of the entity. Through the above steps, the present invention can convert a piece of text into a vector of fixed dimension, which captures the semantic information of the words in the text. This process can be formally expressed as formula (7), and the specific algorithm flow is as follows:

[0130]

[0131] Input: entity description text set Texts, trained Word2Vec model M

[0132] Output: Vector representation of the text set Texts Vectors

[0133] l: Initialize the vector set Vectors to be empty

[0134] 2: for each text text∈Texts do

[0135] 3: Initialization vector The vector dimension is the same as M

[0136] 4: Word segmentation to obtain word sequence W text

[0137] 5: Initialize word count n = 0

[0138] 6: for each word w∈W text do

[0139] 7: if w is in the vocabulary of model M then

[0140] 8: v text =v text +M(w)

[0141] 9: n = n + 1

[0142] 10: end if

[0143] 11: end for

[0144] 12: if n≠0then

[0145] 13: v text =v text / n

[0146] 14: end if

[0147] 15: Add v text To Vectors

[0148] 16: end for

[0149] 17: return Vectors

[0150] In step S103, the weight of each edge in the network security intelligence knowledge graph is calculated, and a random walk sequence is constructed based on the weight of each edge, and a vectorized representation of the network security intelligence knowledge graph node is obtained according to the random walk sequence.

[0151] Among them, random walk is a method of exploring in a graph, which starts from a node and selects nodes adjacent to the node for traversal according to a certain random strategy to generate a node sequence of a certain length.

[0152] Specifically, based on the network security knowledge graph obtained in the aforementioned step S101, the weight of each edge between entity nodes in the network security knowledge graph is calculated, and a random walk sequence is constructed according to the random walk method and weight. The generation of this sequence reflects the path of exploring the graph starting from the node, and then the vectorized representation of the network security intelligence knowledge graph node is calculated according to the random walk sequence.

[0153] For example, for a graph G = (V, E), V is a set of nodes, representing entities of network security intelligence, and E is a set of edges, representing the association relationship between entities. Each edge (u, v) ∈ E has a weight Weight uv , which represents the strength of the relationship between entities u and v. The goal of the random walk is to generate a node sequence walk(u) = [c1, c2, …, c I ], where c i ∈V the i-th node, and the length of the sequence is l. The generation process of this sequence reflects the path of exploring the graph starting from node u, where the node selection at each step is based on the neighbor nodes of the current node.

[0154] In order to construct the above random walk sequence, we first consider a basic completely random walk process based on the idea of ​​DeepWalk. Assume that the current node is v, the previous node is t, and the next node is x. In this process, the next node c in the sequence i The choice depends on the current node c i-1and its neighbors. This selection mechanism can be formalized as the following probability distribution:

[0155]

[0156] Among them, P(c i =x|c i-1 =v) means that given the current node c i-1 If v is the next node c i is the probability of x; N(v) represents the set of neighbor nodes of node v; |N(v)| represents the number of neighbor nodes of node v.

[0157] However, formula (8) only considers a fixed random walk strategy, that is, selecting neighbor nodes as the next node with equal probability. Although this strategy can capture the local structure of the graph, it may also cause some important global information to be ignored during the walk, so it has certain limitations. Therefore, the present invention draws on Node2vec on the basis of formula (8), uses the second-order random walk method, and optimizes the probability formula.

[0158] Specifically, Node2vec adjusts the calculation method of transition probability so that it not only depends on the structure of the graph, but also considers the relative position relationship between the previous and current nodes in the walk path. The calculation formula of transition probability is:

[0159]

[0160] Among them, α pq (t, x) is an adjustment factor based on the relative position between the node t visited in the previous step and the neighbor node x currently considered, which is used to adjust the exploration strategy of the walk. The denominator is a normalization operation to ensure that the sum of the transition probabilities is 1. Adjustment factor α pq The definition of (t, x) is shown in formula (10). In formula (10), d(t, x) represents the shortest path distance between the previous node t and the next node x, corresponding to different wandering strategies:

[0161] (1) If d(t, x) = 0, this means that the walk will return to the node t that has just been visited, and the model will strengthen the neighborhood exploration of the current node.

[0162] (2) If d(t, x) = 1, this indicates that node x is a direct neighbor of the previous node t. The model is not biased towards exploring local or global structures and maintains the natural exploration pattern of random walks.

[0163] (3) If d(t, x) = 2, this indicates that node x is not a direct neighbor of the previous node t, but both node x and node t are connected to node v. The model will explore more distant nodes, which helps to learn the global structure of the graph.

[0164]

[0165] By adjusting p and q, Node2vec can balance between breadth-first search (BFS) and depth-first search (DFS). When both p and q are 1, the model achieves a completely random walk, similar to DeepWalk. When p is set to a small value, i.e., p < min(q, 1), the model tends to perform local exploration, similar to the BFS strategy. On the contrary, when p is set to a large value, i.e., p > max(q, 1), the model tends to perform global exploration, similar to the DFS strategy.

[0166] On this basis, the present invention also considers the edge weights to optimize the random walk strategy. After introducing the edge weights, the generation process of the node sequence will more precisely reflect the closeness of the relationships between network security intelligence entities.

[0167] Optionally, in some embodiments, the preset walk strategy is:

[0168]

[0169] where P(c i =x|c i-1 =v, c i-2 =t) is the transition probability, α pq (t, x) is the relative position adjustment factor based on the previously visited node t and the currently considered neighbor node x, Weight vx is the weight of the edge (v, x), Σ n∈N(v) α pq (t, n) is the sum of all neighbor nodes of v, and Weight vn is the weight of the edges corresponding to all neighbor nodes of v.

[0170] Specifically, when considering the transition from the current node v to the next node x, the transition probability depends not only on the relative position adjustment factor α pq (t, x), but also on the weight Weight vx of the edge (v, x). This consideration makes the orientation of the walk process in the graph affected not only by the structure but also by the strength of the relationships between entities. In Equation (11), Weight vx reflects the edge weight, i.e., the strength of the relationship between nodes v and x. The denominator part sums over all neighbor nodes n of v, considering both the position adjustment factor α pq (t, n) and the corresponding edge weight Weight vn to achieve normalization and ensure that the sum of the transition probabilities is 1.

[0171] Optionally, in some embodiments, the weight of each edge in the network security intelligence knowledge graph is calculated, and a random walk sequence is constructed based on the weight of each edge, including: determining the importance adjustment factor of each edge in the network security intelligence knowledge graph and the node degree adjustment factor of each node, and calculating the weight of each edge in the network security intelligence knowledge graph based on the importance adjustment factor, the node degree adjustment factor and a preset basic weight; based on the weight of each edge in the network security intelligence knowledge graph and a preset walking strategy, a random walk is performed in the network security intelligence knowledge graph to generate a random walk sequence of the network security intelligence knowledge graph.

[0172] Specifically, in order to more accurately reflect the influence of each vulnerability and the importance of resources, the embodiment of the present invention simplifies the connection structure into weight calculation, so that the weight of the edge becomes a probability factor that determines the access path.

[0173] It should be noted that if Figure 4 As shown, Figure 4 This is a schematic diagram of the changes in the knowledge graph connection structure before and after the introduction of supernodes according to a specific embodiment of the present invention, where resources 1 to 7 represent the same series of products provided by the same supplier, and can therefore be represented by a supernode SuperNode1, and similarly SuperNode2 and SuperNode3 can be generated. In the above figure, vulnerability 1 affects resources 1 to 8, and the vulnerability does not affect resources 6 and 8 compared to vulnerability 1, while vulnerability 3 affects resources 7 to 12.

[0174] First-Order Similarity in graph theory and network analysis refers to a method for measuring the similarity of two nodes based on direct connection relationships. Specifically, if two nodes are directly connected, that is, there is an edge between them, then the two nodes are considered to have first-order similarity. This similarity measures the most direct connection mode and reflects the direct adjacency relationship between nodes. If the weight is not considered and only the connection structure of the graph is considered, it can be found that vulnerability 1, vulnerability 2, and vulnerability 3 are all connected to SuperNode1. Although there is a separate field inside the vulnerability entity information to describe specific resource information, the difference in the relationship between SuperNode1 and each vulnerability cannot be seen from the connection structure. Therefore, in the above example, if the weight is not considered, the connection relationship between SuperNode1 and each vulnerability is the same when analyzing it at the first-order similarity level. However, analyzing the structure of the graph before the introduction of supernodes shows that vulnerability 1 and vulnerability 2 are associated with all versions of resource nodes under SuperNode1, while vulnerability 3 is only associated with resource 7 under SuperNode1. Based on the analysis of the original graph structure, it is not difficult to find that the connection relationship between vulnerability 1, vulnerability 2 and this series of resources is closer.

[0175] Second-Order Similarity is a method in graph theory and network analysis to measure the similarity between two nodes based on the node neighborhood structure. Unlike first-order similarity, second-order similarity not only considers whether the nodes are directly connected, but also considers the similarity between the neighbors (or first-order neighbors) of these nodes. Specifically, if the sets of neighbor nodes of two nodes are structurally similar or have similar connection patterns, they are considered to have second-order similarity even if there is no direct edge connecting the two nodes. Figure 4 In the analysis of the similarities between the three vulnerabilities, it can be found that before the introduction of the supernode, Vulnerability 1 and Vulnerability 2 have 6 common neighbors; Vulnerability 3 has 2 common neighbors with Vulnerability 1 and only one common neighbor with Vulnerability 2; therefore, at the level of second-order similarity, Vulnerability 1 and Vulnerability 2 are more similar. However, after the introduction of the supernode, the number of common neighbors between Vulnerability 1 and Vulnerability 2 becomes 1, while the number of common neighbors between Vulnerability 1 and Vulnerability 3 is still 2, which affects the judgment of their similarity.

[0176] Therefore, the embodiment of the present invention fully considers the connection relationship between intelligence entities, and can distinguish the closeness of the connection relationship between different nodes, so that the model can learn potential deeper graph structure features. Figure 4 In the example, without considering the influence of the connection relationship caused by other nodes, the first-order similarity is used to measure the closeness of the node connection relationship. Assuming that vulnerability 1 is the starting node, w1 should be greater than w2 when deciding the next node to visit, that is, the probability of visiting SuperNode1 should be greater than SuperNode2; similarly, assuming that vulnerability 3 is the starting node, w2>w3>w1, that is, the access probability from high to low is SuperNode2, SuperNode3, SuperNode1.

[0177] For example, the embodiment of the present invention adds an "Edge_count" field to the edge of the knowledge graph. For a directed edge from node u to node v, its Edge_count value represents the number of edges from u to v. Among them, for a supernode, its Edge_count is related to the number of resource nodes it is actually associated with. For example, assuming that node u represents a vulnerability node and node v represents an optimized supernode, its Edge_count is the actual number of resources affected by the vulnerability. Figure 4In the example, the Edge_count from vulnerability 1 to SuperNode1 should be 7, and the Edge_count from vulnerability 1 to SuperNode2 should be 1. Therefore, for vulnerability 1, the "importance" of vulnerability 1 to SuperNode1 is greater than that of SuperNode2, and the same is true for other cases. In order to reflect the importance of the relationship between nodes, let the number of connections from node u to v be Edge_count(u, v), then the importance adjustment factor is defined as:

[0178]

[0179] Among them, α is an adjustment parameter, which can be adjusted according to the actual situation. In addition, considering that some resources may be updated and iterated very quickly, when a vulnerability is found, its impact covers more resource nodes than other resources. In order to avoid excessive concentration of weights, the present invention adopts to balance the growth rate and ensure that the adjustment factor can be controlled even if the number of connections is very large.

[0180] Adamic-Adar similarity is a metric used to measure the similarity between two nodes in a social network. The core idea is that the more neighbors two nodes share, the more similar they are, but this similarity is weighted according to the degree of the shared neighbors (that is, the number of connections between neighbors). The smaller the degree of the neighbor node, the greater its contribution to the similarity. This means that if two nodes share many low-degree neighbors, the Adamic-Adar similarity between them will be higher than that of a pair of nodes that share many high-degree neighbors. The formula for calculating Adamic-Adar similarity is as follows:

[0181]

[0182] Among them, N(u) and N(v) are the neighbor sets of nodes u and v respectively, N(u)∩N(v) is the common neighbor set of u and v, and N(c) is the degree of common neighbor c (that is, the number of neighbors of c). In this way, Adamic-Adar similarity not only considers the number of common neighbors between nodes, but also the importance of these common neighbors.

[0183] Referring to Adamic-Adar similarity, the present invention performs degree adjustment of nodes.

[0184] The specific scheme is as follows: for each node v, its "degree adjustment factor" is defined as:

[0185]

[0186] Where degree(v) represents the degree of node v.

[0187] Finally, based on the importance adjustment and node degree adjustment, the importance differences between different intelligence entities are considered and the weight w of the edge from node u to v is calculated.

[0188] Optionally, in some embodiments, the weight of each edge in the network security intelligence knowledge graph is:

[0189] w(u,v)=w relation_type ×degree_adj(v)×importance_adj(u,v)(15)

[0190] Among them, w(u,v) is the weight of each edge in the network security intelligence knowledge graph, w relation_type It is the basic weight setting for the relationship between different intelligence types, degree_adj(v) is the degree adjustment factor of each node, and importance_adj(u,v) is the importance adjustment factor.

[0191] In addition, if you want to focus on the impact of the vulnerability on resources, you can set w relation_CVE_SuperNode The value of is 10, while other types of relationship weights can be set to numbers smaller than 10.

[0192] Optionally, in some embodiments, a vectorized representation of a network security intelligence knowledge graph node is obtained according to a random walk sequence, including: initializing a vector for each node in the network security intelligence knowledge graph; determining positive sample nodes and negative sample nodes according to the random walk sequence based on a preset loss function and context window; training a preset model using positive sample nodes, vectors corresponding to the positive sample nodes, negative sample nodes, and vectors corresponding to the negative sample nodes to obtain a vectorized model for vectorizing the nodes, so as to obtain a vectorized representation of the network security intelligence knowledge graph node through the vectorized model.

[0193] Specifically, after completing the random walk, the present invention obtains a large number of node sequences, which may contain direct and indirect connections between entities in the network security intelligence knowledge graph. In order to mine useful structural information from these connections, the present invention uses a vectorized method to represent these nodes. The present invention draws on the Skip-gram model in the field of natural language processing to mine the potential relationship between words by predicting the probability of word order appearing in a given context. Specifically, facing the word order W = [w1, w2, w3, ..., w n ], the goal of this invention is to optimize the model to improve Pr(w n |w1,w2,…,w n-1) accuracy. On this basis, the present invention applies the idea of ​​the Skip-gram model to the knowledge graph, and regards the node sequence obtained by random walk as a series of "sentences", in which each node is equivalent to a "word" in the sentence. In this way, a model can be constructed to predict the node sequence c1, c2, ..., c that has been traversed. i-1 Based on this, the next node c i Probability of occurrence:

[0194] Pr(ci|(c1,c2,K,c i-1 ))(16)

[0195] By expressing in formula (16), the present invention can transform the problem of predicting the probability of the next node to be generated into the following problem of estimating conditional probability:

[0196] Φ:c∈VαR |V|×d (17)

[0197] According to the previous analysis, the goal of the present invention is to achieve the vectorized representation of nodes, rather than just the probability distribution of node co-occurrence. Therefore, the present invention is to learn the feature representation of nodes, which is defined as Φ, which is a function that maps nodes to vector space:

[0198] Pr(c i |(Φ(c1),Φ(c2),K,Φ(c i-1 ))) (18)

[0199] In large-scale knowledge graphs, especially as the length of random walk sequences increases, the computational complexity of directly calculating this conditional probability is very large, which becomes infeasible. Therefore, the present invention draws on the idea of ​​the Skip-gram model and adopts the following simplified method: instead of using context to predict a word, a word is used to predict its context. The context here refers to the nodes before and after a given node, such as Figure 5 Ultimately, the goal of the present invention is to optimize a model that maximizes the probability of a central node co-occurring with its context nodes, without paying particular attention to the exact position of the node in the context:

[0200]

[0201] where w is the size of the context window, c i-w ,…,c i+w \c i Represents the context node set, including the node at node c i The window size before and after is all nodes within the range of w, but excluding c i itself.

[0202] In order to simplify the calculation of formula (19), the present invention draws on the Skip-Gram model and uses the independence assumption to approximate the conditional probability, that is, when the central node c is known, i In the case of j The probability of occurrence of . Therefore, we can get:

[0203]

[0204] Where Φ(c i ) represents node c i The vector representation of Pr(c j |Φ(c i )) represents the given central node c i Under the condition that the vector representation of j Probability of occurrence.

[0205] Further, similar to the data preprocessing process in step S102 above, the softmax function can be used to calculate the value of each context node c j The conditional probability of occurrence. That is, given the central node c i The vector representation of Φ(c i ), the present invention can predict any context node c j The probability of occurrence. This probability is calculated as follows:

[0206]

[0207] Where |V| is the total number of nodes in the graph, Φ(c j ) T Φ(c i ) is calculated for the central node c i With context node c j The inner product of the vectors between them reflects the similarity between them. The denominator sums the similarity index of all nodes in the graph as context nodes to achieve normalization.

[0208] By minimizing the negative log-likelihood function and using stochastic gradient descent to update the vectorized representation Φ of the nodes, the potential structural relationship between nodes can be captured more accurately. The node vectorization representation method based on graph structure provides a basis for further graph analysis and missing information completion.

[0209] In addition, in large-scale knowledge graphs, the softmax function is directly used to calculate the value of each context node c jThe conditional probability complexity is very high. It can be seen from formula (21) that each calculation needs to traverse all nodes in the entire node set V for normalization. In order to simplify this process and accelerate model training, the present invention introduces a negative sampling method, which approximates the summation process in the denominator by selecting a small number of negative samples (i.e., nodes that do not appear in a specific context), as shown in formula (22):

[0210]

[0211] in, is the sigmoid function, neg represents the number of negative samples, Φ(n) refers to the vector representation of the negative sample nodes randomly selected from the distribution P(V), where P(V) is defined as:

[0212]

[0213] In formula (23), k represents each node in the set V of all nodes in the graph. This formula calculates the probability of node v being selected as a negative sample. The 3 / 4 power processing reduces the probability of high-frequency nodes being selected, so that low-frequency nodes also have a certain chance to be selected as negative samples, making the model learning more balanced.

[0214] Exemplarily, the model algorithm used in the embodiment of the present invention for training the vectorized representation of network security intelligence nodes is as follows:

[0215] Input: node sequence set walk(u), vector representation Φ, context window size w

[0216]

[0217] According to the above-mentioned model algorithm for training the vectorized representation of network security intelligence nodes, the overall process algorithm of the vectorized representation of network security intelligence knowledge graph nodes is as follows:

[0218] Input: Cybersecurity intelligence knowledge graph G(V, E), the context window size w of the central node of the random walk sequence, the number of random walks from each node γ, the length l of each random walk sequence

[0219] Output: vectorized representation matrix of each node:

[0220]

[0221] In step S104, a semantic embedding completion result is obtained according to the numerical vector, and a graph embedding completion result is obtained according to the vectorized representation of the network security intelligence knowledge graph node, and a final missing information completion result is obtained according to the semantic embedding completion result and the graph embedding completion result, so as to perform information completion according to the final missing information completion result.

[0222] Specifically, after realizing the vectorized representation of nodes, the present invention uses cosine similarity to calculate the similarity between entity nodes in the knowledge graph. Cosine similarity determines the similarity between two vectors by measuring the cosine value of the angle between them, and its value range is from -1 to 1. The closer the similarity is to 1, the more similar the two vectors are. For any two entity nodes E of the same type, target and E i , the similarity between its fusion vectors is calculated as follows:

[0223]

[0224] Where, “·” represents the dot product of vectors, |vector(E i )| represents the vector E i It should be noted that similarity calculation is only performed when two entities are of the same type. This ensures that similarity calculation and subsequent missing information completion operations are performed between entities of the same type, thereby ensuring the validity of the completed information.

[0225] According to the calculated similarity, the present invention further selects target The top K entities with the highest similarity are used to complete the missing information. To ensure the accuracy of the completion, only entities with non-empty values ​​in the relevant attribute fields are considered. If the number of entities that meet the conditions is less than K, the actual number will prevail. Here, K is usually set to 10.

[0226] Since the missing attribute value completion of the present invention is mainly aimed at standardized category data, such as the way the supplier exploits the loophole (ADJACENT_NETWORK, LOCAL, Network, Physical), the present invention proposes a set of missing information completion methods based on similarity weighting. The voting weight of each similar entity is determined by their similarity with the target entity, and the formula is as follows:

[0227]

[0228] Among them, C is the set of all possible categories, label i is the category label of the i-th similar entity, and I is the indicator function used to determine whether the category of the similar entity is c. i If c is equal, then I(labeli =c) is 1, that is, similar entity i casts a vote for category c; otherwise, I(label i =c) has a value of 0. i It is the weight calculated based on similarity and normalized to ensure that the sum of the weights is 1. The calculation method is shown in formula (26):

[0229]

[0230] After obtaining the completion results of the semantic embedding model and the graph embedding model respectively, the present invention takes the intersection of the completion results of the two models to further improve the reliability of the completion results. Through the above scheme, the present invention can achieve the filling of security intelligence entity attribute values, thereby improving the effectiveness of network security intelligence data, such as Figure 6 shown.

[0231] According to the missing information completion method based on advanced semantic embedding and graph embedding proposed in the embodiment of the present invention, the obtained network security intelligence data is preprocessed, and each entity text in the preprocessed network security intelligence data is converted into a numerical vector using a pre-trained Word2Vec model; then the weight of each edge in the corresponding network security intelligence knowledge graph is calculated, and a random walk sequence is constructed based on the weight of each edge to obtain a vectorized display of the network security intelligence knowledge graph node; finally, according to the numerical vector and the vectorized display of the network security intelligence knowledge graph node, the semantic embedding completion result and the graph embedding completion result are calculated, and the final missing information completion result is calculated according to the two completion results, and the missing information is completed. Thus, by vectorizing the information in the intelligence data and the knowledge graph, and combining the completion results of the semantic embedding and graph embedding models to improve the reliability of the final completion result, the completion of the missing information of the network security intelligence knowledge graph is achieved, and the auxiliary computing tool can be used to directly process and analyze the complex information in the knowledge graph, thereby improving the computing efficiency and accuracy.

[0232] Next, a missing information completion device based on high-level semantic embedding and graph embedding proposed in an embodiment of the present invention is described with reference to the accompanying drawings.

[0233] Figure 7 It is a block diagram of a missing information completion device based on high-level semantic embedding and graph embedding provided according to an embodiment of the present invention.

[0234] like Figure 7 As shown, the missing information completion device 10 based on high-level semantic embedding and graph embedding includes: an acquisition module 100, a preprocessing module 200, a wandering module 300 and an information completion module 400.

[0235] The acquisition module 100 is used to acquire network security intelligence data and a network security intelligence knowledge graph corresponding to the network security intelligence data;

[0236] A preprocessing module 200 is used to preprocess the network security intelligence data and convert each entity text in the preprocessed network security intelligence data into a numerical vector satisfying a preset dimension using a pre-trained Word2Vec model;

[0237] A walk module 300 is used to calculate the weight of each edge in the network security intelligence knowledge graph, and to construct a random walk sequence based on the weight of each edge, and to obtain a vectorized representation of the network security intelligence knowledge graph node according to the random walk sequence;

[0238] The information completion module 400 is used to obtain a semantic embedding completion result based on a numerical vector, and to obtain a graph embedding completion result based on a vectorized representation of a node in a network security intelligence knowledge graph, and to obtain a final missing information completion result based on the semantic embedding completion result and the graph embedding completion result, so as to complete the information based on the final missing information completion result.

[0239] According to one embodiment of the present invention, the walking module 300 is specifically used to: determine the importance adjustment factor of each edge in the network security intelligence knowledge graph and the node degree adjustment factor of each node, and calculate the weight of each edge in the network security intelligence knowledge graph according to the importance adjustment factor, the node degree adjustment factor and the preset basic weight; based on the weight of each edge in the network security intelligence knowledge graph and the preset walking strategy, perform random walking in the network security intelligence knowledge graph to generate a random walk sequence of the network security intelligence knowledge graph.

[0240] According to one embodiment of the present invention, the walk module 300 is specifically used to: initialize a vector for each node in the network security intelligence knowledge graph; determine the positive sample nodes and the negative sample nodes according to the random walk sequence based on the preset loss function and context window; use the positive sample nodes, the vectors corresponding to the positive sample nodes, the negative sample nodes and the vectors corresponding to the negative sample nodes to train the preset model to obtain a vectorized model for vectorizing the nodes, so as to obtain the vectorized representation of the network security intelligence knowledge graph nodes through the vectorized model.

[0241] According to one embodiment of the present invention, the weight of each edge in the network security intelligence knowledge graph is:

[0242] w(u,v)=w relation_type ×degree_adj(v)×importance_adj(u,v);

[0243] Among them, w(u,v) is the weight of each edge in the network security intelligence knowledge graph, wrelation_type It is the basic weight setting for the relationship between different intelligence types, degree_adj(v) is the degree adjustment factor of each node, and importance_adj(u,v) is the importance adjustment factor.

[0244] According to one embodiment of the present invention, the preset roaming strategy is:

[0245]

[0246] Among them, P(c i =x|c i-1 =v,c i-2 =t) is the transition probability, α pq (t, x) is the relative position adjustment factor between the node t visited in the previous step and the neighbor node x currently considered, and Weight vx is the weight of edge (v, x), Σ n∈N(v) α pq (t, n) is the sum of all v's neighbor nodes, Weight vn is the weight of the edge corresponding to all v's neighbor nodes.

[0247] According to the missing information completion device based on advanced semantic embedding and graph embedding proposed in the embodiment of the present invention, the acquired network security intelligence data is preprocessed, and each entity text in the preprocessed network security intelligence data is converted into a numerical vector using a pre-trained Word2Vec model; then the weight of each edge in the acquired corresponding network security intelligence knowledge graph is calculated, and a random walk sequence is constructed based on the weight of each edge to obtain a vectorized display of the network security intelligence knowledge graph node; finally, according to the numerical vector and the vectorized display of the network security intelligence knowledge graph node, the semantic embedding completion result and the graph embedding completion result are calculated, and the final missing information completion result is calculated according to the two completion results, and the missing information is completed. Thus, by vectorizing the information in the intelligence data and the knowledge graph, and combining the completion results of the semantic embedding and graph embedding models to improve the reliability of the final completion result, the completion of the missing information of the network security intelligence knowledge graph is achieved, and the auxiliary computing tool can be used to directly process and analyze the complex information in the knowledge graph, thereby improving the computing efficiency and accuracy.

[0248] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0249] A memory 801 , a processor 802 , and a computer program stored in the memory 801 and executable on the processor 802 .

[0250] When the processor 802 executes the program, the missing information completion method based on high-level semantic embedding and graph embedding provided in the above embodiment is implemented.

[0251] Furthermore, the electronic device further comprises:

[0252] The communication interface 803 is used for communication between the memory 801 and the processor 802 .

[0253] The memory 801 is used to store computer programs that can be executed on the processor 802 .

[0254] The memory 801 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0255] If the memory 801, the processor 802 and the communication interface 803 are implemented independently, the communication interface 803, the memory 801 and the processor 802 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0256] Optionally, in a specific implementation, if the memory 801, the processor 802 and the communication interface 803 are integrated on a chip, the memory 801, the processor 802 and the communication interface 803 can communicate with each other through an internal interface.

[0257] The processor 802 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0258] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned missing information completion method based on high-level semantic embedding and graph embedding.

[0259] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned missing information completion method based on high-level semantic embedding and graph embedding.

[0260] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0261] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0262] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A missing information completion method based on high-level semantic embedding and graph embedding, characterized in that: The following steps are involved: Acquire network security intelligence data and a network security intelligence knowledge graph corresponding to the network security intelligence data; Preprocessing the network security intelligence data, and using a pre-trained Word2Vec model to convert each entity text in the preprocessed network security intelligence data into a numerical vector that meets a preset dimension; Calculate the weight of each edge in the network security intelligence knowledge graph, and construct a random walk sequence based on the weight of each edge, and obtain a vectorized representation of the network security intelligence knowledge graph node according to the random walk sequence; A semantic embedding completion result is obtained according to the numerical vector, and a graph embedding completion result is obtained according to the vectorized representation of the nodes of the network security intelligence knowledge graph, and a final missing information completion result is obtained according to the semantic embedding completion result and the graph embedding completion result, so as to perform information completion according to the final missing information completion result.

2. The method according to claim 1, characterized in that The calculating the weight of each edge in the network security intelligence knowledge graph and constructing a random walk sequence based on the weight of each edge includes: Determine an importance adjustment factor of each edge and a node degree adjustment factor of each node in the network security intelligence knowledge graph, and calculate a weight of each edge in the network security intelligence knowledge graph according to the importance adjustment factor, the node degree adjustment factor and a preset basic weight; Based on the weight of each edge in the network security intelligence knowledge graph and a preset walking strategy, a random walk is performed in the network security intelligence knowledge graph to generate a random walk sequence of the network security intelligence knowledge graph.

3. The method according to claim 2, characterized in that The step of obtaining a vectorized representation of the network security intelligence knowledge graph node according to the random walk sequence includes: Initializing a vector for each node in the network security intelligence knowledge graph; Based on a preset loss function and a context window, determining positive sample nodes and negative sample nodes according to the random walk sequence; The preset model is trained using the positive sample nodes, the vectors corresponding to the positive sample nodes, the negative sample nodes and the vectors corresponding to the negative sample nodes to obtain a vectorized model for vectorizing the nodes, so as to obtain the vectorized representation of the network security intelligence knowledge graph nodes through the vectorized model.

4. The method according to claim 2, characterized in that: The weight of each edge in the cybersecurity intelligence knowledge graph is: w(u,v)=w relation_type ×degree_adj(v)×importance_adj(u,v); Among them, w(u,v) is the weight of each edge in the network security intelligence knowledge graph, w relation_type It is the basic weight setting for the relationship between different intelligence types, degree_adj(v) is the degree adjustment factor of each node, and importance_adj(u,v) is the importance adjustment factor.

5. The method according to claim 2, characterized in that: The preset wandering strategy is: Among them, P(c i =x|c i-1 =v,c i-2 =t) is the transition probability, α pq (t, x) is the relative position adjustment factor between the node t visited in the previous step and the neighbor node x currently considered, and Weight vx is the weight of edge (v, x), Σ n∈N(v) α pq (t, n) is the sum of all v's neighbor nodes, Weight vn is the weight of the edge corresponding to all v's neighbor nodes.

6. A missing information completion device based on high-level semantic embedding and graph embedding, characterized in that: include: An acquisition module, used to acquire network security intelligence data and a network security intelligence knowledge graph corresponding to the network security intelligence data; A preprocessing module, used to preprocess the network security intelligence data, and use a pre-trained Word2Vec model to convert each entity text in the preprocessed network security intelligence data into a numerical vector that meets a preset dimension; A walk module, used to calculate the weight of each edge in the network security intelligence knowledge graph, and to construct a random walk sequence based on the weight of each edge, and to obtain a vectorized representation of the network security intelligence knowledge graph node according to the random walk sequence; An information completion module is used to obtain a semantic embedding completion result based on the numerical vector, and to obtain a graph embedding completion result based on the vectorized representation of the network security intelligence knowledge graph node, and to obtain a final missing information completion result based on the semantic embedding completion result and the graph embedding completion result, so as to complete the information according to the final missing information completion result.

7. The device according to claim 6, characterized in that The calculating the weight of each edge in the network security intelligence knowledge graph and constructing a random walk sequence based on the weight of each edge includes: Determine an importance adjustment factor of each edge and a node degree adjustment factor of each node in the network security intelligence knowledge graph, and calculate a weight of each edge in the network security intelligence knowledge graph according to the importance adjustment factor, the node degree adjustment factor and a preset basic weight; Based on the weight of each edge in the network security intelligence knowledge graph and a preset walking strategy, a random walk is performed in the network security intelligence knowledge graph to generate a random walk sequence of the network security intelligence knowledge graph.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the missing information completion method based on high-level semantic embedding and graph embedding as described in any one of claims 1 to 5.

9. A computer storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the missing information completion method based on high-level semantic embedding and graph embedding as described in any one of claims 1 to 5.

10. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, is used to implement the missing information completion method based on high-level semantic embedding and graph embedding as described in any one of claims 1 to 5.

Images