A network data leakage detection method based on topic mining and element discovery

Abstract

A network data leakage detection method based on topic model and element discovery belongs to the technical field of network data leakage detection and discovery. The method includes the steps of text preprocessing, text topic mining, topic element discovery, data leakage detection, etc. Text preprocessing is used to perform word segmentation, stop word removal, part-of-speech filtering, etc. on the text to obtain a text word vector representation. Text topic mining builds a topic model, and obtains the "text-topic" distribution and "topic-word" distribution through training. Topic element discovery builds an element discovery model to mine the words most relevant to the topic. Data leakage detection determines whether data leakage occurs by mining the topic to which the text belongs and matching the topic element words. The method uses a topic model to mine text topics, realizes the understanding of text content at the semantic level, builds a node probability transfer matrix with topic preference, realizes topic-oriented element discovery, and improves the accuracy of data leakage detection.

Description

Technical Field

[0001] The present invention relates to a network data leakage detection method based on topic model and element discovery, belonging to the field of network data leakage detection and discovery. Background Art

[0002] Network data leakage refers to the intentional or unintentional release of sensitive information of an organization or individual into an untrusted network environment. In recent years, with the rapid development of information technology, digitization, networking, and intelligence have been deeply integrated into all aspects of social life. Data has become an important production factor and asset. Whether it is government agencies, enterprises, institutions or individuals, they have accumulated a large amount of data. Once these data are leaked, it will seriously affect national security, social public interests, and the legitimate rights and interests of citizens, legal persons and other organizations. How to timely discover and effectively prevent network data leakage has become an important research area in the field of network information security.

[0003] Traditional data leakage detection can be mainly divided into matching-based methods and statistical-based methods. Matching-based methods usually use keyword or phrase matching, as well as pre-defined rules based on regular expressions. Such methods are relatively simple and difficult to detect deformed data. Statistical-based methods select statistical features of data as the basis for judging whether data leakage has occurred. Commonly used statistical features include word weight information (such as word frequency, relative word frequency, TF-IDF value, etc.) and word association information (such as word co-occurrence, mutual information, dependency, etc.). Although such methods extract content in general, they pay less attention to specific text content, so it is difficult to meet user needs in terms of detection accuracy.

[0004] With the continuous development of technologies such as data mining and natural language processing, algorithm models such as text mining and machine learning have been widely used in network data leakage detection scenarios. Text mining methods include topic models and graph models. Topic models perform topic mining on texts to obtain the topic distribution of texts, and then determine whether they belong to the category of data leakage at the semantic level. Graph models represent texts as network graphs, and improve the ability to discover key content in texts by modeling the text structure. Machine learning methods include hidden Markov models (HMMs) and deep learning models, which use a large amount of text to train machine learning algorithms. The algorithms extract the features of the text based on context and natural semantics to form a discriminant model. However, machine learning methods need to rely on a large number of high-quality annotated training sets and the models are relatively complex. The detection accuracy will drop sharply when the text is deformed.

[0005] In summary, existing data leakage detection methods are mainly based on predefined detection rules, text statistical features, keyword discovery, etc. However, network data has different expressions in different scenarios. Therefore, the existing methods need to be further improved in terms of detection accuracy and tolerance to changes in detection content. Summary of the invention

[0006] The purpose of the present invention is to propose a network data leakage detection method based on topic mining and element discovery. The method mines the topics of network data and extracts keywords related to the topics, and then determines whether the data belongs to sensitive data leakage, so as to improve the accuracy of network data leakage detection.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A network data leakage detection method based on topic mining and element discovery includes the following steps:

[0009] Step 1: Capture various data packets on the network port and restore their data to obtain a training data set. Then perform preprocessing on the training data set, such as text segmentation, stop word removal, and part-of-speech filtering, to obtain a vector representation of the text in the training data set as the input of the topic mining model and feature discovery model.

[0010] Step 2: Build a topic mining model and use the Gibbs sampling algorithm to estimate the model parameters to obtain the "text-topic" distribution and "topic-word" distribution.

[0011] Step three, build an element discovery model, calculate the word topic distribution similarity based on the "topic-word" distribution to complete the text network diagram construction, design the node transfer probability based on the word topic distribution similarity, word topic expressiveness and random transfer probability, and finally use the random walk algorithm on the text network diagram to obtain the topic importance of the node and complete the topic element discovery.

[0012] Step 4: Network data leakage detection. The data packets on the network port are captured and restored to obtain text data. After preprocessing, the text data is input into the topic mining model to obtain the topic of the text data. Then, keyword matching is used to determine whether the text data contains the elements corresponding to the text topic. If so, it is a data leakage. If not, it is not a data leakage.

[0013] The above-mentioned network data leakage detection method based on topic mining and element discovery. In the first step, the text tokenization refers to splitting the text into word units to obtain a word sequence, which facilitates the subsequent representation and processing of the text. Professional tokenization tools such as jieba tokenizer and HanLP can be used. The stop-word removal means filtering out the stop words such as "de", "di", "le" etc. that have no actual meaning in the text to reduce their interference with topic mining. Usually, a stop-word list is adopted, and the words in the stop-word list will be filtered out. The part-of-speech filtering means filtering out the adjectives, adverbs, numeral-classifiers and other words that only play a modifying role in the text to make the expression of the text more concise. Usually, jieba tokenizer etc. is used to implement part-of-speech tagging, and the words with part-of-speech such as adjectives, numerals and classifiers will be filtered out.

[0014] The above-mentioned network data leakage detection method based on topic mining and element discovery. In the second step, the constructed topic model models the text generation process as the following process:

[0015] (1) Extract the probability of the "text-topic" distribution from the prior Dirichlet distribution of the "text-topic" distribution ;

[0016] (2) Generate the topic z of each word in the text m,n ;

[0017] (3) Extract the probability of the "topic-word" distribution from the prior Dirichlet distribution of the "topic-word" distribution ;

[0018] (4) Generate each word w in the text m,n ;

[0019] Therefore, the joint generation probability of the words and topics in the training text set is:

[0020]

[0021] Among them, and are called the Dirichlet distribution normalization coefficients.

[0022] The conditional probability of the joint generation probability is Among them, u=(m,n) is a two-dimensional subscript, corresponding to the nth word in the mth text, means removing the word with subscript u.

[0023]

[0024] The above Gibbs sampling process is:

[0025] (1) Randomly assign a topic z to each word w in each text in the training text set;

[0026] (2) Rescan the training text set and for each word w, calculate the conditional probability Sample it and generate a new topic;

[0027] (3) Repeat step (2) until Gibbs sampling converges;

[0028] (4) Statistical distribution of “text-topic” in training text set “Topic-Word” Distribution

[0029] In step 3, when constructing a text network graph, the constructed feature discovery model defines the words in all texts belonging to the same topic as nodes, defines the topic distribution similarity between words as edges, and then uses a random walk algorithm on the network graph to score the importance of the nodes. When performing a random walk, in addition to considering the topic distribution similarity of the nodes, the transition probability between nodes also additionally defines the random transition probability and the node topic expression power, thereby realizing a topic-oriented random walk, so that the words most relevant to the topic can obtain the highest score.

[0030] The above text network graph is defined as a quadruple G =<V,E,Ω,P> :

[0031] V={v i |v i is the word contained in the text}. The word i in the text is represented by the node v in the text network graph i .

[0032] E={e ij |e ij is node v i With node v j The edge between i ∈V,v j ∈V}. Node v i With node v j The edges between are defined as directed edges.

[0033] Ω={ω ij |ω ij It is edge ij The weight of ij ∈E}. Weight ω ij Reflects the node v i With node v j The degree of correlation between

[0034] Using node v i With node v j A measure of the similarity of the word-topic distributions of corresponding words.

[0035] P = {p i |p i is node v i The importance weight, v i ∈V}. The importance weight of the node is calculated using the random walk algorithm on the graph.

[0036] The above “word-topic” distribution describes the probability of words belonging to different topics. Its mathematical calculation formula is as follows:

[0037]

[0038] The above “word-topic” distribution similarity is measured using KL divergence. i and node v j , the “word-topic” distribution similarity calculation formula is as follows:

[0039]

[0040] The above node v i and node v j The weight between them is the inverse of the "word-topic" distribution similarity and adding 1 to both the numerator and denominator:

[0041]

[0042] Since the fully connected graph will lead to a large amount of calculation, the above text graph filters the topic distribution similarity of words by setting a threshold ∈. i With node v j If the topic distribution similarity of the corresponding words is lower than the set threshold ∈, the similarity is ignored and the node v is considered i With node v j There is no edge between them. The above text graph will i With σ u =[1 / k,1 / k,…,1 / k] KL divergence as threshold ∈, if σ i With σ j If the KL divergence value is greater than the threshold, it is considered that σ i With σ j Negative correlation, ignore the similarity and delete node v i With node v j The edge between.

[0043] ∈ i =D KL (σ i ||σu )

[0044] The node transfer probability of the above random walk algorithm is:

[0045] Λ'=(1-d)(Λ+D)+dE

[0046] Among them, Λ, D and E are the topic distribution similarity of nodes (i.e., the weight Ω of the edge in the network graph), random transition probability and the topic expressiveness of words, respectively.

[0047] The above random transition probability D is mainly used to ensure that any two nodes in the network graph are connected, so that the random walk can converge to a unique value, which is defined as:

[0048]

[0049] in, is an n-dimensional column vector, indicating uniform distribution on all nodes, where n is the number of nodes in the network graph:

[0050]

[0051] vector is an n-dimensional row vector used to identify nodes with degree 0:

[0052]

[0053] The above-mentioned word topic expressiveness E is a variable used to express the tendency of random walk, which is measured by the information entropy of the "word-topic" distribution. The calculation formula of the information entropy of the "word-topic" distribution is as follows:

[0054]

[0055] The softmax function is used to convert it into probability, and the final word topic expression matrix E is:

[0056]

[0057] The above random walk process is:

[0058] (1) Randomly give an initial node importance vector p(0);

[0059] (2) Calculate the weight p(1) of each node after the first walk according to the random walk iterative formula; the iterative formula is as follows:

[0060] p(t+1)=d*(Ω+D)p(t)+(1-d)*e

[0061] Where e is an n-dimensional row vector whose value is:

[0062]

[0063] (3) Repeat step (2) until the random walk converges;

[0064] (4) At this time, the weight p of each node is the topic importance of each node.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The network data leakage detection method described in the present invention is a method based on topic mining and element discovery, including the steps of text preprocessing, text topic mining, topic element discovery, data leakage detection, etc. Text preprocessing is used to perform word segmentation, stop word removal, part-of-speech filtering, etc. on the text to obtain a text word vector representation. Text topic mining constructs a topic model, and obtains the "text-topic" distribution and "topic-word" distribution through training. Topic element discovery constructs an element discovery model to mine the words most relevant to the topic. Data leakage detection determines whether data leakage occurs by mining the topic to which the text belongs and matching the topic element words. The method uses a topic model to perform text topic mining, realizes the understanding of text content at the semantic level, builds a node probability transfer matrix with topic preference, realizes topic-oriented element discovery, and improves the accuracy of data leakage detection.

[0067] The method obtains the topics described by the text data set through the topic mining model, and then obtains the topic elements of each topic through the element discovery model. Compared with the existing methods, the present invention uses the topic model to mine text topics, realizes the description of text content from the semantic level, and improves the understanding of text content in the process of data leakage detection. In addition, the element discovery model proposed in the present invention realizes topic-oriented element discovery by constructing a node probability transfer matrix with topic preference, thereby improving the accuracy and recall rate of topic element discovery. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is the technical roadmap of the present invention.

[0069] Figure 2 It is a structural diagram of the topic mining model of the present invention.

[0070] Figure 3 It is the technical roadmap of the element discovery model of the present invention. DETAILED DESCRIPTION

[0071] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0072] like Figure 1 As shown, the present invention provides a network data leakage detection method based on topic mining and element discovery, comprising the following steps:

[0073] Step 1: First, capture and restore the data packets of the network port, and save each text as an independent TXT file. Then preprocess the text data set, use jieba word segmentation to complete the text segmentation, use the open source stop word list to complete the stop word filtering, and filter out words with adjectives, numerals, quantifiers, and locatives according to the part-of-speech tagging results in the jieba word segmentation process, and only retain words with part-of-speech tags of "x: non-morpheme words, vn: noun verbs, n: nouns" to complete the part-of-speech filtering. Finally, assign a digital number to each word as its index ID to obtain the vocabulary corresponding to the text set.

[0074] For each text, the number of occurrences of each word is counted, so that the text is represented as a text vector. For example, the vector (1,2,0,0,1) means that the word numbered 1 in the vocabulary appears once in the text, the word numbered 2 appears twice, the word numbered 5 appears once, and the words numbered 3 and 4 do not appear. On this basis, the entire text set is represented as a two-dimensional matrix, in which each row is a vector representation of a text, and the number of rows represents the size of the text set.

[0075] Step 2, such as Figure 2 As shown, the topic model used in the present invention and is a hyperparameter of the model. The model parameter that needs to be estimated is the "topic-word" distribution and the “text-topic” distribution The training process uses the Gibbs sampling algorithm to determine the optimal hyperparameters and The present invention adopts the grid search method to determine the optimal hyperparameter combination, and determines and The value range and search step length are and The model training process is as follows:

[0076] (1) Randomly assign a topic z to each word w in each text in the training text set;

[0077] (2) Rescan the training text set and for each word w, calculate the conditional probability Sample it and generate a new topic; conditional probability The calculation formula is:

[0078]

[0079] (3) Repeat step (2) until Gibbs sampling converges;

[0080] (4) Statistical distribution of “text-topic” in training text set “Topic-Word” Distribution

[0081] After the model training is completed, the perplexity index is used to evaluate the model performance. The calculation formula is as follows:

[0082]

[0083] Among them, N d is the number of words in the text d, p(w dn )=∑ z p(w n |z)p(z|d) is the probability of generating word n ​​in text d. The smaller the perplexity, the better the topic model performance.

[0084] Step three, such as Figure 3 As shown, the feature discovery model constructed by the present invention first needs to construct a text network diagram, and the word i contained in the text set is represented by a node v in the text network diagram. i , node v i With node v j The topic similarity between word i and word j is represented by , and the weight of the edge is measured by the topic similarity between word i and word j. The formula for calculating the topic similarity between word i and word j is as follows:

[0085] (1) Based on the “topic-word” distribution Calculate the word-topic distribution

[0086]

[0087] (2) Using KL divergence to measure the similarity of two “word-topic” distributions

[0088]

[0089] Because D KL (σ i ||σ j )≠D KL (σ j ||σ i ), so node v i and node v j The edges between are directed edges, and the graph formed is a directed weighted graph. Since the fully connected graph will lead to a large amount of calculation, the similarity of the topic distribution of words is filtered by setting a threshold ∈. i With node v j If the topic distribution similarity of the corresponding words is lower than the set threshold ∈, the similarity is ignored and the node v is consideredi With node v j There is no edge between them. i With σ u =[1 / k,1 / k,…,1 / k] KL divergence as threshold ∈, if σ i With σ j If the KL divergence value is greater than the threshold, it is considered that σ i With σ j Negative correlation, ignore the similarity and delete node v i With node v j The edge between.

[0090] ∈ i =D KL (σ i ||σ u )

[0091] Compute Node v i With node v j The weight calculation formula of the edge is as follows:

[0092]

[0093] Secondly, we need to calculate the node topic expressiveness, which is measured by the information entropy of the “word-topic” distribution. The calculation formula of the information entropy of the “word-topic” distribution is as follows:

[0094]

[0095] The softmax function is used to convert it into probability, and the final word topic expression matrix is:

[0096]

[0097] The node transfer probability of the random walk process is obtained as:

[0098] Λ'=(1-d)(Λ+D)+dE

[0099] Among them, Λ, D and E are the topic distribution similarity of nodes (i.e., the weight Ω of the edge in the network graph), random transition probability and topic expressiveness of words respectively. The random transition probability D is mainly used to ensure that any two nodes in the network graph are connected, so that the random walk can converge to a unique value, which is defined as:

[0100]

[0101] in, is an n-dimensional column vector, indicating uniform distribution on all nodes, where n is the number of nodes in the network graph:

[0102]

[0103] vector is an n-dimensional row vector used to identify nodes with degree 0:

[0104]

[0105] Finally, the random walk process is:

[0106] (1) Randomly give an initial node importance vector p(0);

[0107] (2) Calculate the weight p(1) of each node after the first walk according to the random walk iterative formula; the iterative formula is as follows:

[0108] p(t+1)=d*(Ω+D)p(t)+(1-d)*e

[0109] Where e is an n-dimensional row vector whose value is:

[0110]

[0111] (3) Repeat step (2) until the random walk converges;

[0112] (4) At this time, the weight p of each node is the topic importance of each node.

[0113] Step 4: Network data leakage detection. First, use step 1 to preprocess the text of the real-time network data packets, and then input the text into the topic model for topic mining. The process is as follows:

[0114] (1) Randomly assign a topic z to each word w in each text in the real-time input text;

[0115] (2) Rescan the text and for each word w, calculate the conditional probability Sampling it generates a new topic; at this time, the conditional probability The topic model obtained by training in step 2 is stable and unchanged.

[0116] (3) Repeat step (2) until Gibbs sampling converges;

[0117] (4) Obtain text themes through statistics.

[0118] After obtaining the text topic, the topic element words corresponding to the topic are taken out from the element discovery model in step three and matched with the text. If the topic element words appear in the text, it is determined that data leakage has occurred; if not, no data leakage is found.

Claims

1. A network data leakage detection method based on topic model and element discovery, characterized in that: The following steps are involved: Step 1: Capture the data packets on the network port and restore their data to obtain a training data set; then perform text segmentation, stop word removal, and part-of-speech filtering preprocessing on the training data set to obtain a vector representation of the text in the training data set as the input of the topic mining model and the feature discovery model; Step 2: Build a topic mining model, use the Gibbs sampling algorithm to estimate the model parameters, and obtain the "text-topic" distribution and "topic-word" distribution; Step 3: Build an element discovery model. According to the "topic-word" distribution calculation, the word topic distribution similarity is obtained to complete the text network diagram construction. The node transfer probability is designed based on the word topic distribution similarity, word topic expressiveness and random transfer probability. Finally, a random walk algorithm is used on the text network diagram to obtain the topic importance of the node and complete the topic element discovery. Step 4: Network data leakage detection. The data packets on the network port are captured and restored to obtain text data. After preprocessing, the text data is input into the topic mining model to obtain the topic of the text data. Then, keyword matching is used to determine whether the text data contains the elements corresponding to the text topic. If so, it is a data leakage. If not, it is not a data leakage.

2. The network data leakage detection method based on topic mining and element discovery according to claim 1 is characterized by: In the step 1, the data set is segmented into word units to obtain word sequences; then stop words without practical meaning are filtered out to reduce their interference with topic mining; finally, words that only play a modifying role in the text are filtered out.

3. A network data leakage detection method based on topic mining and element discovery according to claim 1, characterized in that: The topic mining process in step 2 is as follows: (1) Calculate the joint generation probability of words and topics in the training text set: in, and is the normalized coefficient of Dirichlet distribution, is the Dirichlet distribution prior parameter, m represents the mth text, k represents the kth topic, is the "text-topic" distribution probability, is the probability of "topic-word" distribution, w is the word, z is the topic; (2) Joint generation probability The conditional probability of Among them, u = (m, n) is a two-dimensional subscript, corresponding to the nth word in the mth text, It means removing the words with subscript u; Among them, α k yes The topic k corresponds to the hyperparameter value, Indicates the number of times the remaining words belong to topic k after removing the nth word from the mth text; β t yes The hyperparameter value corresponding to the word t in , It indicates the number of times that the remaining words are t after removing the nth word in the mth text from the words with topic k; (3) Gibbs sampling to estimate the probability of "text-topic" distribution "Topic-word" distribution probability The Gibbs sampling process is as follows: (3.1) Randomly assign a topic z to each word w in each text in the training text set; (3.2) Rescan the training text set and for each word w, according to the conditional probability Sample it and generate a new topic; (3.3) Repeat step (3.2) until Gibbs sampling converges; (3.4) Statistics on the “text-topic” distribution of the training text set "Topic-Word" Distribution 4. A network data leakage detection method based on topic mining and element discovery according to claim 1, characterized in that: In step 3, the feature discovery model defines the text network graph quadruple G=<V,E,Ω,P> , to represent the text; V={v i |v i is the word contained in the text}; word i in the text is represented by node v in the text network diagram i ; E={e ij |e ij is node v i With node v j The edge between i ∈V,v j ∈V}; node v i With node v j The edges between are defined as directed edges; Ω={ω ij |ω ij It is edge ij The weight of ij ∈E}; weight ω ij Reflects the node v i With node v j The degree of correlation between Using node v i With node v j A measure of the similarity of the "word-topic" distribution of the corresponding words; P = {p i |p i is node v i The importance weight, v i ∈V}; the importance weight of the node is calculated using the random walk algorithm on the graph.

5. The network data leakage detection method based on topic mining and element discovery according to claim 1 is characterized in that: In step 3, the "word-topic" distribution σ is used w Describe the probability of words belonging to different topics: Among them, σ w，k represents the probability of the kth topic appearing in word w, represents the probability of word w appearing in the kth topic, and uses KL divergence to calculate the topic similarity D between word i and word j KL (σ i ‖σ j ) to measure: So we get node v i With node v j The weight of the edge between them:

6. A network data leakage detection method based on topic mining and element discovery according to claim 1, characterized in that: In step 3, the node transfer probability of the random walk algorithm is: Λ′=(1-d)(Λ+D)+dE Among them, Λ is the topic distribution similarity of the node, that is, the weight Ω of the edge in the network graph; D is the random transition probability, and E is the topic expression matrix of the word; The random transition probability D is defined as: in, is an n-dimensional column vector, indicating uniform distribution on all nodes, where n is the number of nodes in the network graph: vector is an n-dimensional row vector used to identify nodes with degree 0:

7. A network data leakage detection method based on topic mining and element discovery according to any one of claims 1 or 5, characterized in that: In step 3, the above-mentioned word theme expressiveness E is a variable used to express the tendency of random walk, and the information entropy H (σ i ) for measurement; The softmax function is used to convert it into probability, and the final word topic expression matrix E is:

8. The network data leakage detection method based on topic mining and element discovery according to claim 1 is characterized in that: The process of random walking in step 3 is: (1) Randomly give an initial node importance vector p(0); (2) Calculate the weight p(1) of each node after the first walk according to the random walk iterative formula; the iterative formula is as follows: p(t+1)=d*(Ω+D)p(t)+(1-d)*e Where e is an n-dimensional row vector whose value is: (3) Repeat step (2) until the random walk converges; (4) At this time, the weight p of each node is the topic importance of each node.

Images