Network security detection method based on multi-source heterogeneous data fusion
By collecting and integrating multi-source heterogeneous data, extracting factor features and building a multi-factor model, the problem of insufficient accuracy of network security behavior detection in the prior art is solved, and efficient monitoring and protection of network security behavior is achieved.
Patent Information
- Application Number
- CN202410974840.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-19
AI Technical Summary
The existing technology is difficult to effectively integrate multi-source heterogeneous data, resulting in insufficient accuracy of network security behavior detection and targeted protection measures.
By collecting telecommunications, user behavior and social media data, data cleaning, factor feature extraction and multi-factor model construction are carried out to achieve accurate detection and dynamic protection of network security behavior.
It improves the accuracy of network security behavior detection and targeted protection measures, and achieves efficient monitoring and protection of network security behavior.
Smart Images

Figure CN118984231A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a technology in the field of network security, and specifically to a network security detection method based on multi-source heterogeneous data fusion. Background Art
[0002] With the rapid development of information technology, the fusion application of multi-source heterogeneous data can break data silos, realize extensive information sharing and efficient interaction, and thus improve the accuracy and effectiveness of early warning. Summary of the invention
[0003] In view of the above shortcomings of the prior art, the present invention proposes a network security detection method based on multi-source heterogeneous data fusion, which integrates multi-source heterogeneous data, accurately extracts characteristic factors from different data sources, and selects factors that are highly interpretable to network security behavior. Based on these factors, a multi-factor model of network security behavior is constructed to achieve security detection of network security behavior, and provide targeted security protection measures accordingly.
[0004] The present invention is achieved through the following technical solutions:
[0005] The present invention relates to a network security detection method based on multi-source heterogeneous data fusion. After collecting and integrating heterogeneous data sources such as telecommunications, user behavior, social media, and financial data, factor features are extracted and a factor feature library is constructed. A multi-factor model and a detection model are constructed and trained, and finally online testing and dynamic adjustment are performed.
[0006] The present invention relates to a system for implementing the above method, comprising: a data cleaning unit, a feature extraction unit, a model training unit and an online detection unit, wherein: the data cleaning unit performs data preprocessing according to data source information to obtain cleaned data results, the feature extraction unit performs factor feature extraction according to the cleaned data results to obtain feature factors, the model training unit constructs a multi-factor model according to the feature factor results, and performs initialization learning to obtain preliminary model results, and the online detection unit performs online testing and dynamic adjustment according to the preliminary model results to obtain final safety detection results. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Figure 1 It is a flow chart of the present invention;
[0008] Figure 2 is a flow chart of step S1;
[0009] Figure 3 is a flow chart of step S3;
[0010] Figure 4 Schematic diagram of the multi-factor model. DETAILED DESCRIPTION
[0011] like Figure 1 As shown, a network security detection method based on multi-source heterogeneous data fusion involved in this embodiment includes:
[0012] S1. Data integration and fusion: Collect and integrate heterogeneous data sources such as telecommunications, user behavior, social media, and financial data, including:
[0013] 1.1. Collect and integrate data: Collect multi-source heterogeneous data such as telecommunications data, user behavior data, social media data, and financial data. Build a unified database architecture to accommodate and integrate data of various formats and structures.
[0014] 1.2. Data preprocessing: Thoroughly clean, deduplicate and convert the original data, adopt data standardization and metadata management technology, and establish a unified data model and metadata standard.
[0015] 1.3. Data mining and association analysis: Use knowledge graph and graph analysis technology to build a relationship graph between users and entities, and deeply explore the implicit relationships and social network structures between data.
[0016] S2. Extracting factor features and building a factor feature library, including:
[0017] 2.1. Extracting factor features: Specifically, construct an adjacency matrix A for the connections between different users (nodes) in the network, which satisfies: when there is an edge between nodes i and j, then A ij =w ij , where w ij is the weight of the edge. When there is no edge, then A ij =0; then construct the degree matrix D in the form of a diagonal matrix, the elements on the diagonal are the degree of each node, that is, the sum of the weights of the edges connected to the node; then the adjacency matrix and degree matrix satisfy: Laplace matrix L = DA, perform eigenvalue decomposition on the Laplace matrix L to obtain eigenvectors and eigenvalues; select the eigenvectors corresponding to the k smallest non-zero eigenvalues of the Laplace matrix to form the eigenvector matrix U. The rows or columns in the eigenvector matrix U are used as points in the new feature space, and then the K-means algorithm is used to cluster these points to obtain factor features, in which each element represents the community classification to which the user belongs. Nodes belonging to the same community are closely connected to each other, while nodes in different communities are sparsely connected.
[0018] 2.2. The features extracted according to step 2.1 are expressed in the form of weighted sum of bases, and a set of bases is obtained as the factor feature library, specifically: Features Among them: the base {e 1 ,e2 ,....,e n ...}, Where: α and β are eigenvectors extracted from the eigenvector matrix U, e i The eigenvector represents the basis in the factor feature library, a i Represents the weight coefficient of the basis vector, which is used to construct features.
[0019] 2.3. Constructing factor feature library: For different feature vectors α in feature space, their explanatory power is different. Assume that the features with strong explanatory power constitute a linear subspace V1 of feature space V. Let {e 1 ,e 2 ,....,e m} constitute V1, then for any eigenvector For different factors, we can use the factor fusion method to obtain α=F(α 1 ,α 2 ,α 3 ...,α n ), where: F is the IC weighting method, Thus, factor fusion is performed. i Representation factor α i Safety detection accuracy possibility.
[0020] S3, such as Figure 3 As shown, a multi-factor model and a detection model are constructed, including:
[0021] 3.1. Constructing a multi-factor model: E(r i )=β i,1 F i,1 +β i,2 F i,2 +β i,3 F i,3 +......+β i,n F i,n +ε i , where: F i,k represents the possibility of the i-th individual being in danger under factor feature k itself, β i,k Represents the exposure of the i-th individual to factor feature k, compares the model prediction value and the true value, and calculates ε i is the residual, E(r i ) represents the possibility that an individual’s cybersecurity behavior may cause danger.
[0022] 3.2. Construct the detection model, specifically: Objective function: y = f(ΣE(r i )+k); by adding the regularization term θ i |c i-c 0,i | to reduce the possibility of overfitting. And through the variable step size c ij To learn. Introduce hyperparameter θ = {θ 1 ,θ 2 ,...,θ m} to further increase the depth of learning.
[0023] 3.3、Train the detection model, specifically: input training set sample T = {(x 1 ,y 1 ),(x 2 ,y 2 ),…,(x m ,y m )}, the maximum number of iterations is T, and the loss function is set to L; construct the initialization learning model: Where: c = {c 1 ,c 2 ,…,c m}; Perform iterative training and calculate the gradient for each sample After that, calculate the number of regression leaf nodes and the best fitting value Where: c = {c 1 ,c 2 ,…,c m}, θ={θ 1 ,θ 2 ,…,θ m}; and update the model to: That is, we get the model expression f(x)=f T (x).
[0024] 3.4. For the objective function obtained in step 3.3, design the following supervised learning algorithm:
[0025] In supervised learning, the labeled training data is used to train the model to distinguish between normal transactions and fraudulent behaviors through the function iteration algorithm: V k+1 (s)=max a (R (s,a) +γ∑ s′ P(s′|s,a)V k (s′)), where: V k+1 (s) represents the fraud probability estimate of the value function of state s at the k+1th iteration, R (s,a) It means that state s gets function feedback after getting a factor a. P(s′|s,a) means the transition probability of state s transferring to state s′ after getting factor a.
[0026] Choose the optimal strategy: π * (s) = arg a max a (R (s,a) +γ∑ s′ P(s′|s,a)V k (s′)), where: V * (s) represents the optimal value function, which represents the expected return of taking the optimal strategy in state s, π * (s) represents the optimal strategy, which indicates the optimal action to be taken in state s.
[0027] 3.5. Estimation of coefficients of multi-factor model: In addition to the above algorithm, the coefficients of multi-factor model can also be directly estimated by the following method: β = (F T F) -1 F T E, where: element F in matrix F i,k It represents the possibility of the i-th individual being in danger in the factor feature k itself, the vector E represents the possibility of N individual network security behaviors causing danger, and the vector β represents the exposure of individuals to different factor features.
[0028] After specific actual experiments, in an environment where the laboratory network is configured with a bandwidth of 100Mbps, the above device / method is started / run with 50 simulated user nodes performing 100 network behaviors (including web browsing, social media interaction and financial transactions) every day. The experimental data that can be obtained are: user behavior data includes timestamp 2024-07-18 10:00:00, behavior type web browsing and data volume 500KB; social media interaction data includes timestamp 2024-07-18 10:05:00, behavior type social media comment and data volume 200KB; financial transaction data includes timestamp 2024-07-18 10:10:00, behavior type online payment and data volume 1000KB. Through these data, the multi-factor model is verified, and the average error rate between the predicted value and the actual value is calculated to be 5%. Based on the error analysis results, improvement suggestions are proposed to increase the sample size and optimize the feature extraction method.
[0029] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.
Claims
1. A network security detection method based on multi-source heterogeneous data fusion, characterized in that: By collecting and integrating heterogeneous data sources such as telecommunications, user behavior, social media, and financial data, we extract factor features and build a factor feature library, build and train multi-factor models and detection models, and finally conduct online testing and dynamic adjustments.
2. The network security detection method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The collection and integration of heterogeneous data sources of telecommunications, user behavior, social media, and financial data specifically includes: 1.1) Collect and integrate data: collect telecommunications data, user behavior data, social media data and financial data, and build a unified database architecture to accommodate and integrate data of various formats and structures; 1.2) Data preprocessing: Thoroughly clean, remove duplicates and convert the format of the original data, adopt data standardization and metadata management technology, and establish a unified data model and metadata standard; 1.3) Data mining and association analysis: Use knowledge graphs and graph analysis techniques to build relationship graphs between users and entities, and deeply explore the implicit relationships and social network structures between data.
3. The network security detection method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The extracting factor features and constructing a factor feature library specifically includes: 2.1) Extracting factor features, specifically: constructing an adjacency matrix A for the connections between different users (nodes) in the network, which satisfies: when there is an edge between nodes i and j, then A ij =w ij , where w ij is the weight of the edge. When there is no edge, then A ij =0; then construct a degree matrix D in the form of a diagonal matrix, where the elements on the diagonal are the degree of each node, that is, the sum of the weights of the edges connected to the node; then the adjacency matrix and the degree matrix satisfy: Laplace matrix L = DA, perform eigenvalue decomposition on the Laplace matrix L to obtain eigenvectors and eigenvalues; select the eigenvectors corresponding to the k smallest non-zero eigenvalues of the Laplace matrix to form an eigenvector matrix U, use the rows or columns in the eigenvector matrix U as points in the new feature space, and then use the K-means algorithm to cluster these points to obtain factor features, where each element represents the community classification to which the user belongs, and the nodes within the same community are closely connected to each other, while the connections between nodes in different communities are relatively sparse; 2.2) The features extracted according to step 2.1 are expressed in the form of weighted sum of bases, and the obtained set of bases is used as the factor feature library, specifically: Features Among them: the basis {e1,e2,....,e n ...}, Where: α and β are eigenvectors extracted from the eigenvector matrix U, e i The eigenvector represents the basis in the factor feature library, a i Represents the weight coefficient of the basis vector, which is used to construct features; 2.3) Constructing factor feature library: For different feature vectors α in feature space, their explanatory power is different. Assuming that the features with strong explanatory power constitute a linear subspace V1 of feature space V, let {e1,e2,....,e m } constitute V1, then for any eigenvector For different factors, we can get α=F(α1,α2,α3...,α n ), where: F is the IC weighting method, Thus, factor fusion is performed, where w i Representation factor α i Safety detection accuracy possibility.
4. The network security detection method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The construction and training of the multi-factor model and the detection model specifically include: 3.1) Constructing a multi-factor model: E(r i )=β i,1 F i,1 +β i,2 F i,2 +β i,3 F i,3 +......+β i,n F i,n +ε i , where: F i,k represents the possibility of the i-th individual being in danger under factor feature k itself, β i,k Represents the exposure of the i-th individual to factor feature k, compares the model prediction value and the true value, and calculates ε i is the residual, E(r i ) represents the possibility that an individual's cybersecurity behavior may cause danger; 3.2) Construct the detection model, specifically: Objective function: y = f(ΣE(r i )+k); by adding the regularization term θ i |c i -c 0,i | to reduce the possibility of overfitting, and through the variable step size c ij To learn, we introduce hyperparameters θ = {θ1, θ2, ..., θ m }, to further increase the depth of learning; 3.3) Training the detection model, specifically: Input training set samples T = {(x1, y1), (x2, y2), …, (x m ,y m )}, the maximum number of iterations is T, and the loss function is set to L; construct the initialization learning model: Where: c={c1,c2,…,c m }; Perform iterative training and calculate the gradient for each sample After that, calculate the number of regression leaf nodes and the best fitting value Where: c={c1,c2,…,c m }, θ={θ1,θ2,…,θ m }; and update the model to: That is, we get the model expression f(x)=f T (x); 3.4) Design a supervised learning algorithm for the objective function obtained in step 3.3: In supervised learning, the labeled training data will be used to train the model to distinguish between normal transactions and fraudulent behaviors through the function iteration algorithm: V k+1 (s)=max a (R (s,a) +γ∑ s' P(s'|s,a)V k (s')), where: V k+1 (s) represents the fraud probability estimate of the value function of state s at the k+1th iteration, R (s,a) It means that state s obtains function feedback after obtaining a factor a, and P(s'|s,a) indicates the transition probability of state s transferring to state s' after obtaining factor a; Choose the optimal strategy: π * (s) = arg a max a (R (s,a) +γ∑ s' P(s'|s,a)V k( s')), where: V * (s) represents the optimal value function, which represents the expected return of taking the optimal strategy in state s, π * (s) represents the optimal strategy, which indicates the optimal action to be taken in state s; 3.5) Estimation of multi-factor model coefficients: In addition to the above algorithms, the multi-factor model coefficients can also be directly estimated by the following methods: Among them: the element F in the matrix F i,k It represents the possibility of the i-th individual being in danger in the factor feature k itself, the vector E represents the possibility of N individual network security behaviors causing danger, and the vector β represents the exposure of individuals to different factor features.
5. A network security detection system based on multi-source heterogeneous data fusion for implementing any of the methods described in claims 1-4, characterized in that: include: A data cleaning unit, a feature extraction unit, a model training unit and an online detection unit, wherein: the data cleaning unit performs data preprocessing according to the data source information to obtain the cleaned data results; the feature extraction unit performs factor feature extraction according to the cleaned data results to obtain feature factors; the model training unit constructs a multi-factor model according to the feature factor results, and performs initialization learning to obtain preliminary model results; the online detection unit performs online testing and dynamic adjustment according to the preliminary model results to obtain the final security detection results.
