A method and device for detecting threats of original network flows in cyberspace scenarios

Abstract

The present invention discloses a method and device for detecting threats of raw network flows in a cyberspace scenario. The method comprises: obtaining raw network flow data, processing the raw network flow data, and obtaining characteristic information of the raw network flow data; dividing the characteristic information of the raw network flow data, and obtaining training characteristic information of the raw network flow data and test characteristic information of the raw network flow data; using an automatic machine learning model, processing the training characteristic information of the raw network flow data, and obtaining an optimized raw network flow threat detection model; using the optimized raw network flow threat detection model, processing the test characteristic information of the raw network flow data, and obtaining the raw network flow threat detection result. It can be seen that the method of the present invention solves the difficult problems of poor applicability and low degree of automation of the method caused by complex and changeable scenarios, and effectively improves the automatic construction level of the threat detection and analysis model in the cyberspace combat scenario.

Description

Technical Field

[0001] The present invention relates to the field of network security, and in particular to a method and device for detecting original network flow threats in a cyberspace scenario. Background Art

[0002] Among the existing methods of data conversion and data processing in cyberspace, the semantic-based encoding method combines all semantic fields of network flows into one representation with integrity and constant size, but this method has representation uncertainty; the original binary-based encoding method retains the feature order and reduces the reliance on manually designed features, but ignores many complex details in the network flow, which may introduce a lot of interference.

[0003] Among the existing methods for constructing cyberspace scenarios, methods based on machine learning technology and deep neural network technology can build a single model and optimize the model. However, for the construction of cyberspace for multiple scenarios and different applications, there are problems such as low adaptability of the method, poor model effect, and low degree of automation.

[0004] In the actual problem of constructing cyberspace combat scenarios, due to the large differences between different cyberspaces and different requirements for specific cyberspaces during construction, the previous methods will introduce many problems, including:

[0005] (1) The semantic-based encoding method does not preserve the order of the message option fields in each network flow, and the field encoding method needs to be manually determined, resulting in inconsistent feature ordering among network flows processed by this method, which in turn makes the encoding representation unsuitable for tasks such as device identification.

[0006] (2) The raw binary-based encoding method ignores many complex details in the network flow, including the changing length of the network flow and the differences between different protocols. This will result in inconsistent encoding lengths for each network flow and the encoding representation results are not interpretable.

[0007] (3) Methods based on machine learning technology and deep neural network technology can only construct and optimize a single manually specified model, resulting in the training model being only applicable to a single specific cyberspace scenario and unable to achieve cross-cyberspace migration. The model has a low level of automation when it is constructed and has poor adaptability to scenarios in actual applications. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a method and device for detecting threats in a cyberspace scenario based on a raw network flow data representation model, and to perform data conversion and data processing on the network flow in the cyberspace, so that the encoded representation has integrity, consistency and scalability, and the representation is standardized and has a constant size. In addition, by using an automatic machine learning model, the automatic selection of the best model and hyperparameters for a specific cyberspace can be achieved, and the degree of automation of model construction can be improved. The automatic machine learning model can also realize the fusion construction of multiple models for a specific cyberspace, so that the final model has high environmental adaptability.

[0009] In order to solve the above technical problems, the first aspect of the embodiment of the present invention discloses a method for detecting threats of original network flows in a cyberspace scenario, the method comprising:

[0010] S1, obtaining original network flow data, wherein the original network flow data includes a plurality of network flow data;

[0011] The original network flow data includes normal flow data, DDos attack network flow data, and WebAttack attack network flow data; the network flow data is a binary data packet;

[0012] S2, processing the original network flow data to obtain feature information of the original network flow data;

[0013] S3, dividing the feature information of the original network flow data to obtain training feature information of the original network flow data and test feature information of the original network flow data;

[0014] S4, using a preset automatic machine learning model to process the training feature information of the original network flow data to obtain an optimized original network flow threat detection model;

[0015] S5, using the optimized original network flow threat detection model, processing the test feature information of the original network flow data to obtain an original network flow threat detection result.

[0016] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the original network flow data to obtain feature information of the original network flow data includes:

[0017] S21, preprocessing the original network flow data to obtain standardized original network flow data;

[0018] S22, sampling the standardized original network flow data to obtain N subgraphs, where N is a positive integer;

[0019] S23, performing scale normalization processing on the N sub-graphs to obtain N sub-graphs of the same size;

[0020] S24, performing feature extraction on the N subgraphs of the same size to obtain feature information of the original network flow data.

[0021] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the preprocessing of the original network flow data to obtain standardized original network flow data includes:

[0022] S211, performing data alignment on the original network flow data to obtain alignment data information;

[0023] S212, performing internal data padding on the aligned data information to obtain padded data information;

[0024] S213, performing data representation on the padded data information to obtain standardized original network flow data.

[0025] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the extracting features of the N subgraphs of the same size to obtain feature information of the original network flow data includes:

[0026] S241, using a preset feature extraction model, extracting features from any sub-graph to obtain sub-graph feature information;

[0027] The feature extraction model expression is:

[0028]

[0029] In the formula, t represents time, ω represents angular frequency, x() represents subgraph, τ represents delay, is the sub-graph feature information, * indicates taking conjugate;

[0030] S242, using a preset filtering model to filter the sub-image feature information Filter to obtain the sub-image feature information after noise reduction and edge preservation

[0031] S243, performing feature fusion on the sub-graph feature information after denoising and edge preservation of the N sub-graphs to obtain feature information of the original network flow data.

[0032] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the preset filtering model expression is:

[0033]

[0034] In the formula, W() represents the atomic graph feature information, represents the feature information of the sub-image after filtering, m represents the output pixel point, n represents the input pixel point, s represents the set rectangular frame area, which is used to traverse the sub-image, W is the two-dimensional template, and are two Gaussian functions, representing the spatial domain kernel and the pixel domain kernel respectively, M m The specific expression is:

[0035]

[0036] and The specific expression is:

[0037]

[0038]

[0039] In the formula, a and b represent the horizontal and vertical coordinates of the input pixel, i and j represent the coordinates of the center of the box, and σ s , σ r Represents the standard deviation of the Gaussian function.

[0040] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the feature fusion of the sub-graph feature information after denoising and edge preservation of the N sub-graphs to obtain the feature information of the original network flow data includes:

[0041] S2431, using a VGG19 network to process the sub-image feature information after denoising and edge preservation of the N sub-images, to obtain N first sub-image feature information;

[0042] S2432, clustering the N first subgraph feature information using an agglomerative hierarchical clustering method to obtain N cluster feature information;

[0043] S2433: Using a feature fusion model, perform feature fusion on the N cluster feature information to obtain feature information of the original network flow data.

[0044] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the method for calculating the feature fusion model includes:

[0045] Obtain cluster feature information X and cluster feature information Y to be fused;

[0046] Performing feature projection on the cluster feature information X and the cluster feature information Y to obtain projected cluster feature information X' and projected cluster feature information Y';

[0047] Processing the projected cluster feature information X' and the projected cluster feature information Y' to obtain a projection vector a1 and a projection vector b1;

[0048] The cluster feature information X, the cluster feature information Y, the projection vector a1 and the projection vector b1 are processed to obtain fused feature information of the cluster feature information X and the cluster feature information Y.

[0049] A second aspect of an embodiment of the present invention discloses an original network flow threat detection device in a cyberspace scenario, the device comprising:

[0050] An acquisition module, used for acquiring original network flow data, wherein the original network flow data includes a plurality of network flow data;

[0051] The original network flow data includes normal flow data, DDos attack network flow data, and WebAttack attack network flow data; the network flow data is a binary data packet;

[0052] A first processing module, used for processing the original network flow data to obtain feature information of the original network flow data;

[0053] A second processing module is used to divide the characteristic information of the original network flow data to obtain training characteristic information of the original network flow data and test characteristic information of the original network flow data;

[0054] A third processing module is used to process the training feature information of the original network flow data using a preset automatic machine learning model to obtain an optimized original network flow threat detection model;

[0055] The fourth processing module is used to use the optimized original network flow threat detection model to process the test feature information of the original network flow data to obtain the original network flow threat detection result.

[0056] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the processing of the original network flow data to obtain feature information of the original network flow data includes:

[0057] S21, preprocessing the original network flow data to obtain standardized original network flow data;

[0058] S22, sampling the standardized original network flow data to obtain N subgraphs, where N is a positive integer;

[0059] S23, performing scale normalization processing on the N sub-graphs to obtain N sub-graphs of the same size;

[0060] S24, performing feature extraction on the N subgraphs of the same size to obtain feature information of the original network flow data.

[0061] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the preprocessing of the original network flow data to obtain standardized original network flow data includes:

[0062] S211, performing data alignment on the original network flow data to obtain alignment data information;

[0063] S212, performing internal data padding on the aligned data information to obtain padded data information;

[0064] S213, performing data representation on the padded data information to obtain standardized original network flow data.

[0065] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the extracting features of the N subgraphs of the same size to obtain feature information of the original network flow data includes:

[0066] S241, using a preset feature extraction model, extracting features from any sub-graph to obtain sub-graph feature information;

[0067] The feature extraction model expression is:

[0068]

[0069] In the formula, t represents time, ω represents angular frequency, x() represents subgraph, τ represents delay, is the sub-graph feature information, * indicates taking conjugate;

[0070] S242, using a preset filtering model, filtering the sub-image feature information W(t,ω) to obtain the sub-image feature information after noise reduction and edge preservation

[0071] S243, performing feature fusion on the sub-graph feature information after denoising and edge preservation of the N sub-graphs to obtain feature information of the original network flow data.

[0072] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the preset filtering model expression is:

[0073]

[0074] In the formula, W() represents the atomic graph feature information, represents the feature information of the sub-image after filtering, m represents the output pixel point, n represents the input pixel point, s represents the set rectangular frame area, which is used to traverse the sub-image, W is the two-dimensional template, and are two Gaussian functions, representing the spatial domain kernel and the pixel domain kernel respectively, M m The specific expression is:

[0075]

[0076] and The specific expression is:

[0077]

[0078]

[0079] In the formula, a and b represent the horizontal and vertical coordinates of the input pixel, i and j represent the coordinates of the center of the box, and σ s , σ r Represents the standard deviation of the Gaussian function.

[0080] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the feature fusion of the sub-graph feature information after denoising and edge preservation of the N sub-graphs to obtain the feature information of the original network flow data includes:

[0081] S2431, using a VGG19 network to process the sub-image feature information after denoising and edge preservation of the N sub-images, to obtain N first sub-image feature information;

[0082] S2432, clustering the N first subgraph feature information using an agglomerative hierarchical clustering method to obtain N cluster feature information;

[0083] S2433: Using a feature fusion model, perform feature fusion on the N cluster feature information to obtain feature information of the original network flow data.

[0084] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the method for calculating the feature fusion model includes:

[0085] Obtain cluster feature information X and cluster feature information Y to be fused;

[0086] Performing feature projection on the cluster feature information X and the cluster feature information Y to obtain projected cluster feature information X' and projected cluster feature information Y';

[0087] Processing the projected cluster feature information X' and the projected cluster feature information Y' to obtain a projection vector a1 and a projection vector b1;

[0088] The cluster feature information X, the cluster feature information Y, the projection vector a1 and the projection vector b1 are processed to obtain fused feature information of the cluster feature information X and the cluster feature information Y.

[0089] The third aspect of the present invention discloses another original network flow threat detection device in a cyberspace scenario, the device comprising:

[0090] A memory storing executable program code;

[0091] a processor coupled to the memory;

[0092] The processor calls the executable program code stored in the memory to execute part or all of the steps in the original network flow threat detection method in the cyberspace scenario disclosed in the first aspect of the embodiment of the present invention.

[0093] The fourth aspect of the present invention discloses a computer-storable medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the original network flow threat detection method in the cyberspace scenario disclosed in the first aspect of an embodiment of the present invention.

[0094] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0095] (1) The method of the present invention does not need to perform semantic extraction on the original network flow, so that all the information of the original network flow can be retained, while ensuring that the feature order in each data representation is the same, thereby ensuring the integrity and consistency of the representation.

[0096] (2) The present invention focuses on the length differences between different network flows and the encoding differences between different protocols, which can make the data representations of different network flows comparable, consistent and interpretable.

[0097] (3) The present invention has the advantage of fast network flow data processing speed, and can convert an average of 1.5 million data packets per minute under a single thread; at the same time, it has the advantage of being able to collect and encode dynamic network flows online, and also being able to encode network flows offline. Therefore, the present invention can be applied to the construction of network flow threat detection and analysis models in a variety of different cyberspace combat scenarios.

[0098] (4) The advantage of the method of the present invention is that it does not need to rely on manual screening of features for model training, and can train a multi-model fusion model for a single network space construction problem, thereby achieving automatic and rapid parameter optimization and outputting the optimal fusion model. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0100] Figure 1 It is a flowchart of a method for detecting threats of an original network flow in a cyberspace scenario disclosed in an embodiment of the present invention;

[0101] Figure 2 It is a schematic diagram of an original network flow threat detection and analysis model in another cyberspace scenario disclosed in an embodiment of the present invention;

[0102] Figure 3 It is a flow chart of the original network flow data representation model disclosed in the embodiment of the present invention;

[0103] Figure 4 It is a schematic diagram of representing the original network flow data disclosed in the embodiment of the present invention;

[0104] Figure 5 is the neural network architecture of the automatic machine learning model disclosed in the embodiment of the present invention;

[0105] Figure 6 It is a multi-layer stacking strategy of the automatic machine learning model disclosed in the embodiment of the present invention;

[0106] Figure 7 It is a schematic diagram of an original network flow threat detection device in a cyberspace scenario disclosed in an embodiment of the present invention;

[0107] Figure 8 It is a schematic diagram of another original network flow threat detection device in a cyberspace scenario disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0108] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0109] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0110] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0111] The present invention discloses a method and device for detecting threats of raw network flows in a cyberspace scenario. The method comprises: obtaining raw network flow data, processing the raw network flow data, and obtaining characteristic information of the raw network flow data; dividing the characteristic information of the raw network flow data, and obtaining training characteristic information of the raw network flow data and test characteristic information of the raw network flow data; using an automatic machine learning model, processing the training characteristic information of the raw network flow data, and obtaining an optimized raw network flow threat detection model; using the optimized raw network flow threat detection model, processing the test characteristic information of the raw network flow data, and obtaining the raw network flow threat detection result. It can be seen that the method of the present invention solves the difficult problems of poor applicability and low degree of automation of the method caused by complex and changeable scenarios, and effectively improves the automatic construction level of the threat detection and analysis model in the cyberspace combat scenario. The following are detailed descriptions.

[0112] Embodiment 1

[0113] See also Figure 1 , Figure 1 1 is a flow chart of a method for detecting threats of an original network flow in a cyberspace scenario disclosed in an embodiment of the present invention. Figure 1 The original network flow threat detection method described in the cyberspace scenario can be used in the field of network security, such as intrusion detection and defense, and preventing information theft. Figure 1 As shown, the original network flow threat detection method in the cyberspace scenario may include the following operations:

[0114] S1, obtaining original network flow data, wherein the original network flow data includes a plurality of network flow data;

[0115] The original network flow data includes normal flow data, DDos attack network flow data, and WebAttack attack network flow data; the network flow data is a binary data packet;

[0116] S2, processing the original network flow data to obtain feature information of the original network flow data;

[0117] S3, dividing the feature information of the original network flow data to obtain training feature information of the original network flow data and test feature information of the original network flow data;

[0118] Optionally, the division may be performed in a ratio of 7:3, that is, the training feature information of the original network flow data accounts for 70%, and the test feature information of the original network flow data accounts for 30%.

[0119] S4, using a preset automatic machine learning model to process the training feature information of the original network flow data to obtain an optimized original network flow threat detection model;

[0120] S5, using the optimized original network flow threat detection model, processing the test feature information of the original network flow data to obtain an original network flow threat detection result.

[0121] Optionally, the processing the original network flow data to obtain characteristic information of the original network flow data includes:

[0122] S21, preprocessing the original network flow data to obtain standardized original network flow data;

[0123] S22, sampling the standardized original network flow data to obtain N subgraphs, where N is a positive integer;

[0124] S23, performing scale normalization processing on the N sub-graphs to obtain N sub-graphs of the same size;

[0125] Normalizing the scale of the N sub-graphs without changing the structure of each sub-graph, and filling the sub-graphs to obtain N sub-graphs of the same size;

[0126] S24, performing feature extraction on the N subgraphs of the same size to obtain feature information of the original network flow data.

[0127] Optionally, the preprocessing the original network flow data to obtain standardized original network flow data includes:

[0128] S211, performing data alignment on the original network flow data to obtain alignment data information;

[0129] S212, performing internal data padding on the aligned data information to obtain padded data information;

[0130] S213, performing data representation on the padded data information to obtain standardized original network flow data.

[0131] Optionally, the extracting features from the N subgraphs of the same size to obtain feature information of the original network flow data includes:

[0132] S241, using a preset feature extraction model, extracting features from any sub-graph to obtain sub-graph feature information;

[0133] The feature extraction model expression is:

[0134]

[0135] Where t represents time, ω represents angular frequency, x() represents sub-image, τ represents delay, W(t,ω) is the sub-image feature information, and * represents conjugation;

[0136] S242, using a preset filtering model, filtering the sub-image feature information W(t,ω) to obtain the sub-image feature information after noise reduction and edge preservation

[0137] S243, performing feature fusion on the sub-graph feature information after denoising and edge preservation of the N sub-graphs to obtain feature information of the original network flow data.

[0138] Optionally, the preset filtering model expression is:

[0139]

[0140] In the formula, W() represents the atomic graph feature information, represents the feature information of the sub-image after filtering, m represents the output pixel point, n represents the input pixel point, s represents the set rectangular frame area, which is used to traverse the sub-image, W is the two-dimensional template, and are two Gaussian functions, representing the spatial domain kernel and the pixel domain kernel respectively, M m The specific expression is:

[0141]

[0142] G σs and G σr The specific expression is:

[0143]

[0144]

[0145] In the formula, a and b represent the horizontal and vertical coordinates of the input pixel, i and j represent the coordinates of the center of the box, and σ s , σ r Represents the standard deviation of the Gaussian function.

[0146] Optionally, the filtering process includes:

[0147] Set the initial threshold δ th The target interval is [α, β], where 0≤α<β≤max{S(m,n)}, S(m,n) is the time-frequency spectrum;

[0148] Divide the target interval [α, β] into L equal parts;

[0149] The initial threshold δ th Set to α, according to The step size is incremented and iterated to obtain the threshold

[0150] Utilizing Thresholds Calculate the feature information of the sub-image after filtering at each iteration

[0151]

[0152] Among them, S(m,n) represents the atomic graph feature information, represents the feature information of the sub-image after filtering, and a and b represent the time and frequency parameters in the threshold;

[0153] Calculate the difference between each iteration and the sub-graph feature information obtained in the previous iteration i1=1,2,…, i1 is the number of iterations;

[0154] The difference Perform curve fitting to obtain curve C. When curve C has an inflection point, the corresponding threshold is the optimal threshold.

[0155] The optimal threshold is used to perform optimal threshold filtering on the atomic graph feature information S(m,n) to obtain filtered sub-graph feature information.

[0156] Optionally, the step of performing feature fusion on the sub-graph feature information after denoising and edge preservation of the N sub-graphs to obtain the feature information of the original network flow data includes:

[0157] S2431, using a VGG19 network to process the sub-image feature information after denoising and edge preservation of the N sub-images, to obtain N first sub-image feature information;

[0158] S2432, clustering the N first subgraph feature information using an agglomerative hierarchical clustering method to obtain N cluster feature information;

[0159] S2433: Using a feature fusion model, perform feature fusion on the N cluster feature information to obtain feature information of the original network flow data.

[0160] Optionally, the calculation method of the feature fusion model includes:

[0161] Obtain cluster feature information X and cluster feature information Y to be fused;

[0162] Performing feature projection on the cluster feature information X and the cluster feature information Y to obtain projected cluster feature information X' and projected cluster feature information Y';

[0163] Processing the projected cluster feature information X' and the projected cluster feature information Y' to obtain a projection vector a1 and a projection vector b1;

[0164] The cluster feature information X, the cluster feature information Y, the projection vector a1 and the projection vector b1 are processed to obtain fused feature information of the cluster feature information X and the cluster feature information Y.

[0165] The calculation method of the feature fusion model is specifically as follows:

[0166] Get the cluster feature information X and cluster feature information Y to be fused; X is an n′×m′ dimensional matrix, Y is a p×m′ dimensional matrix, m′ represents the number of samples, n′ and p represent the dimensions of the two features, project the two matrices to 1 dimension for linear representation, corresponding to the projection vectors a1 and b1 respectively, then the feature matrix after projection becomes:

[0167]

[0168] The purpose is to maximize the correlation coefficient between X' and Y', so as to obtain the projection vectors a1 and b1 when the correlation coefficient is maximum, that is:

[0169]

[0170] Before projection, the data is standardized. The purpose of standardization is to make the mean of the data 0 and the variance 1. We can get:

[0171]

[0172]

[0173]

[0174] D(X)=cov(X,X)=E(X T X)

[0175] D(Y)=cov(Y,Y)=E(Y T Y)

[0176] cov(X,Y)=E(XY T ),cov(Y,X)=E(YX T )

[0177] S XX =cov(X,X), then the solution objective is transformed into:

[0178]

[0179] Step 1: Calculate the variance S of X and Y XX and S YY , the covariance S of XY and YX XY =S YX T ;

[0180] Step 2: Calculate the matrix

[0181] Step 3: Solve the singular values ​​of M′ to obtain the largest singular value and its preceding and following singular vectors u, v;

[0182] Step 4: The projection vectors a1 and b1 of X and Y are:

[0183]

[0184] Step 5: The fusion feature vector Z of X and Y is

[0185] Optionally, the preset automatic machine learning model is a graph convolution model based on a multiple attention mechanism:

[0186] Let N = 1, 2, ..., n represent the numbers of all nodes, and the feature identifiers of all nodes are X∈R d , x i is the feature of the i-th node, A∈R N×N Represents the adjacency matrix, and the weight of the edge between node i and node j is represented by A ij Any node will collect information from its neighboring nodes, analyze and update the collected information. If each node has similar characteristics to its neighbors, each node can be identified by the average value of its neighboring nodes. Therefore, for all nodes, the update method is:

[0187]

[0188] Since the relationship between each node and its neighboring nodes is different and there are differences in strength, appropriate weighting can be performed according to the strength of the connection between each node and its neighboring points to obtain a more reasonable representation:

[0189]

[0190] The way to process the aggregated nodes is to use a linear transformation matrix W∈R d×h It is used to represent the aggregate features of nodes in the h-dimensional space. The formula is:

[0191] δ(X*W)=δ(AXW)

[0192] Where δ() is the activation function.

[0193] In practical applications, in order to fully consider the characteristics of the node itself, the following formula is often used to update the node:

[0194]

[0195] X * =(A+I)X

[0196] Update the adjacency matrix and degree matrix to obtain:

[0197]

[0198] in, is the updated adjacency matrix, is the updated degree matrix. The matrix is ​​a diagonal matrix. The degree of each vertex represents the information it contains. The degree of each node is defined as the weighted sum of its edges.

[0199] Different nodes have different numbers of edges and different edge weights. Some nodes have more edges and more feature values ​​after aggregation. This problem can be solved by normalizing the self-loop information from neighboring nodes. Before aggregation, it is:

[0200]

[0201] After aggregation:

[0202]

[0203] Next, the information transmitted by neighbor node j is standardized:

[0204]

[0205] The input and output relationship of the normalized graph convolutional model is:

[0206]

[0207] There are two commonly used attention mechanisms, one is the hard attention mechanism and the other is the soft attention mechanism. The working principle of the soft attention mechanism is to take a set of node features As the input of the layer, N is the number of nodes, F is the dimension, and its output is a set of new node features, expressed as Using the weight matrix W∈R F×F′ Perform a parameterized linear transformation on each node, and then use the self-attention mechanism a:R F′×F′ → R is applied to each node to calculate the attention coefficient:

[0208]

[0209] Where: e ij Indicates the importance of the feature of node j to node i. Adding the mask attention mechanism to the graph structure can effectively learn the structural information of the graph and improve the overall performance of the model. The neighborhood of node i is represented by N i Represents, j∈N i . Use the softmax function to normalize all selected j:

[0210]

[0211] The attention mechanism is a single-layer feedforward neural network, which is a weight vector The parameterization of can be nonlinearized using LeakyReLU(·)(α=0.2). The coefficients calculated by the attention mechanism can be expressed as:

[0212]

[0213] Where: T represents the transpose of the matrix, ∪ represents the connection operation. The final feature output of each node is:

[0214]

[0215] The k independent attention mechanisms are cascaded and calculated, and the output features are:

[0216]

[0217] Among them, the normalized attention coefficient calculated for the kth time is Indicates that the weight matrix of the input linear transformation is W k It means that the nonlinear function is averaged as:

[0218]

[0219] Optionally, the preset automatic machine learning model is a deep learning knowledge distillation model, which is:

[0220] Set up the teacher model and the student model;

[0221] The teacher model is a YOLOv5l model; the student model is a YOLOv5s model;

[0222] Setting the temperature coefficient T, we get:

[0223]

[0224] In the formula, p i is the output probability of the i-th category, T is the temperature coefficient, z i is the output value of the i-th category, j is the number of classification categories;

[0225] Use Softmax function to calculate distillation loss Loss soft :

[0226] Loss soft =T 2 *KLdiv(P S ,P T )

[0227] Where P T is the probability distribution of the teacher model, P S is the probability distribution of the student model, KLdiv() is the KL divergence;

[0228] The student model loss Loss is calculated by the difference between the student model and the true label hard for:

[0229] Loss hard =H(P S ,y gt )

[0230] In the formula, y gt represents the true label, H(P S ,y gt ) indicates P S ,y gt The cross entropy of

[0231] The objective function of knowledge distillation is set as:

[0232] Loss = αLoss soft +(1-α)Loss hard

[0233] In the formula, Loss is the total loss, Losssoft is the distillation loss, Loss hard is the student model loss and α is the weight.

[0234] Embodiment 2

[0235] See also Figure 2 , Figure 2 1 is a flow chart of another original network flow threat detection method in a cyberspace scenario disclosed in an embodiment of the present invention. Figure 2 The original network flow threat detection method described in the cyberspace scenario can be used in the field of network security, such as intrusion detection and defense, and preventing information theft. Figure 2 As shown, the original network flow threat detection method in the cyberspace scenario may include the following operations:

[0236] In this embodiment, a model of a method for detecting threat of original network flow in a cyberspace scenario is (FlowDAModel, FlowDAM), which is a threat detection and analysis model based on a unified description language that can adapt to multiple types of rapidly changing original network flows in a cyberspace combat scenario. It can quickly extract network flow data from the original cyberspace and express it in a unified language, and then use the automatic machine learning (Automatic Machine Learning, AutoML) algorithm to quickly and automatically build a network flow threat detection and analysis model. FlowDAM is divided into two parts: original network flow data representation technology (FlowR Technology, FlowRT) and AutoML. FlowRT breaks through the difficulty of unified description of original network flows in the cyberspace battlefield environment, and can expand key technologies such as the construction of cyberspace combat scenario framework; the combination of AutoML algorithm and FlowRT solves the difficult problems of poor applicability and low degree of automation of the method caused by complex and changeable scenarios, and effectively improves the automatic construction level of threat detection and analysis models in cyberspace combat scenarios.

[0237] The construction process of FlowDAM includes raw network flow input, data conversion, data representation, and AutoML process. Among them, data conversion and data representation belong to the raw network flow data representation module, and FlowRT is combined with AutoML to form FlowDAM.

[0238] ① Original network flow input: Users can collect original network flow data sets by themselves, and input the original network flow into the original network flow data representation module for processing.

[0239] ② Original network flow data representation module: The original network flow data representation module represents the data packet in the form of raw binary and aligns the binary data in a way that recognizes the specific semantic structure of the data packet itself. FlowRT uses internal padding to reduce the inconsistency of data representation and retains the order of option features while reducing the impact of inconsistent bit representation. At the same time, FlowRT encodes the semantic structure of the protocol and only requires the maximum length of the protocol, without parsing and representing each field in the protocol.

[0240] ③AutoML: Use the AutoML algorithm to build cyberspace combat scenario models. The model building process can realize automatic feature extraction, hyperparameter tuning, and multi-model fusion construction, making the construction process fast and automated, and the generated model has the characteristics of strong adaptability to multiple scenarios.

[0241] FlowDAM is built in two parts: FlowRT and AutoML. The original network flow data representation module processes and converts the original network flow, and AutoML builds the model of the network space scene. The specific basic principles are as follows. Figure 3 It is a flow chart of the original network flow data representation model disclosed in the embodiment of the present invention.

[0242] 1) FlowRT: The original network flow data representation module represents data in raw binary form and aligns the binary data in a way that identifies the specific semantic structure of the data packet itself. FlowRT uses internal padding to mitigate the inconsistency of data representation, preserving the order of optional features while mitigating the impact of inconsistent bit representation. FlowRT can ensure the integrity of data representation; FlowRT uses internal padding, and each packet has positions for all header types, ensuring that the same number of features are represented in each packet, and each feature has the same meaning. FlowRT is interpretable at the bit level, and FlowRT can be mapped back to the semantic field to better understand the features; FlowRT's representation is standardized, and each bit can only contain -1, 0, and 1. Non-existent header information is represented by -1, which can distinguish between bits that are set to 0 and bits that do not have values; data represented by FlowRT has the same size, and each packet has the same number of features. Use payload to make the number of bytes optional. The data representation format after FlowRT processing is shown in the figure below, Figure 4 The FlowRT representation format of TCP network flow and UDP network flow including the header space of IPV4 protocol, TCP protocol, UDP protocol and ICMP protocol is shown.

[0243] 2) AutoML algorithm: The AutoML algorithm can perform data preprocessing and multi-model fusion training to find the fusion model with the best parameters. In the data processing stage, the AutoML algorithm will automatically identify the feature type and delete irrelevant features, and filter low-frequency features. At the same time, the AutoML algorithm will process missing values ​​and encode specific features; in the model training stage, the AutoML algorithm can test more than 50 models, which can be divided into 6 basic model categories. The basic model categories are evolved from tree methods, deep neural networks, and nearest neighbor algorithms. The AutoML algorithm can preset model training speed parameters and model size parameters. In order to prevent model overfitting, the AutoML algorithm can perform multiple k-fold cross-validation training.

[0244] 3) The neural network architecture of the AutoML algorithm for tabular data consisting of numerical and categorical features is as follows Figure 5 As shown in Figure 1, the layers with learnable parameters are marked in yellow. The multi-layer stacking strategy of the AutoML algorithm is shown in Figure 1. Figure 6 As shown, two stacked layers and different types of base learners are shown here.

[0245] Optionally, to obtain the original network flow data, the user can collect the original network flow data set by himself, and input the original network flow into the original network flow data representation model for processing. The original network flow data are all labeled, such as normal flow, DDos attack, WebAttack attack, etc.

[0246] Optionally, the original network flow data representation model represents the data packet in the original binary form and aligns the binary data in a way that identifies the specific semantic structure of the data packet itself. The original network flow data representation model uses internal padding to alleviate the inconsistency of data representation and preserves the order of optional features while alleviating the impact of inconsistent bit representation.

[0247] The original network flow data representation model can ensure the integrity of data representation; the original network flow data representation model uses internal padding, and each packet has positions for all header types, ensuring that the same number of features are represented in each packet, and each feature has the same meaning. The original network flow data representation model is interpretable at the bit level, and the original network flow data representation model can be mapped back to the semantic field to better understand the features; the representation of the original network flow data representation model is standardized, and each bit can only contain -1, 0, and 1. Non-existent header information is represented by -1, which can distinguish bits that are set to 0 and bits that do not have values; the data represented by the original network flow data representation model has the same size, and each packet has the same number of features. Use payload to make the number of bytes optional. Each network flow data forms a feature vector, and the feature vectors of all network flow data are arranged to form a two-dimensional matrix. The conversion process from original network flow data to two-dimensional matrix data is realized by data processing methods.

[0248] The training feature information of the original network flow data is processed using a preset automatic machine learning model to obtain an optimized original network flow threat detection model.

[0249] Optionally, the automatic machine learning model can perform data preprocessing and multi-model fusion training to find the fusion model with the best parameters. During the data processing stage, the automatic machine learning model will automatically identify the feature type and delete irrelevant features and filter low-frequency features. The automatic machine learning model trains representation vectors for the categorical features in the data and determines the vector length of each feature based on the number of representations of each feature in all input data.

[0250] The automatic machine learning model handles the missing values ​​of continuous features in the data, fills the missing values ​​with the median, encodes the specific features, and then normalizes the data so that the mean is 0 and the standard deviation is 1. Finally, the categorical feature vector is concatenated with the continuous data and put into Figure 4 In the model training phase, the automatic machine learning model can test more than 50 models, which can be divided into 6 basic model categories. The basic model categories are evolved from tree methods, deep neural networks, and nearest neighbor algorithms, which are not limited in the present invention. The automatic machine learning model uses all features and defined algorithms of different types to train multiple basic models, such as Figure 5Model 1, Model 2 to Model n in . Then, multiple models of the same layer interact and update the hyperparameters of the models, where the hyperparameters of the same type of models (such as if Model 1 and Model 2 are both tree models) remain consistent. The goal of updating the model hyperparameters is to gradually improve the accuracy of each round of training. Finally, the stacking of the output layer uses a weighted approach to assign higher weights to high-performance models, and then performs weighted summation of multiple models to generate a fusion of multiple models. The automatic machine learning model can preset model training speed parameters and model size parameters. In order to prevent model overfitting, the automatic machine learning model can perform multiple k-fold cross-validation training.

[0251] K-fold cross validation first randomly divides the data set into k mutually exclusive subsets of equal size, and randomly selects k-1 copies as training sets each time, and the remaining 1 copy as test set. When this round is completed, k copies are randomly selected again to train the data. After several rounds (less than k), the loss function is selected to evaluate the optimal model and parameters.

[0252] Embodiment 3

[0253] See also Figure 7 , Figure 7 Schematic diagram of the structure of an original network flow threat detection device in a cyberspace scenario disclosed in an embodiment of the present invention. Figure 7 The original network flow threat detection device in the described cyberspace scenario can be used in the field of network security, such as intrusion detection and defense, and preventing information theft. Figure 7 As shown, the device comprises:

[0254] S301, an acquisition module, used to acquire original network flow data, wherein the original network flow data includes a plurality of network flow data;

[0255] The original network flow data includes normal flow data, DDos attack network flow data, and WebAttack attack network flow data; the network flow data is a binary data packet;

[0256] S302, a first processing module, configured to process the original network flow data to obtain feature information of the original network flow data;

[0257] S303, a second processing module, used to divide the feature information of the original network flow data to obtain training feature information of the original network flow data and test feature information of the original network flow data;

[0258] S304, a third processing module, used to process the training feature information of the original network flow data using a preset automatic machine learning model to obtain an optimized original network flow threat detection model;

[0259] S305, a fourth processing module, is used to process the test feature information of the original network flow data by using the optimized original network flow threat detection model to obtain an original network flow threat detection result.

[0260] Embodiment 4

[0261] See also Figure 8 , Figure 8 Schematic diagram of the structure of another original network flow threat detection device in a cyberspace scenario disclosed in an embodiment of the present invention. Figure 8 The original network flow threat detection device in the described cyberspace scenario can be used in the field of network security, such as intrusion detection and defense, and preventing information theft. Figure 8 As shown, the device may include:

[0262] A memory S401 storing executable program codes;

[0263] A processor S402 coupled to the memory S401;

[0264] The processor S402 calls the executable program code stored in the memory S401 to execute the steps in the original network flow threat detection method in the cyberspace scenario described in the first or second embodiment.

[0265] Embodiment 5

[0266] An embodiment of the present invention discloses a computer-readable storage medium that stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the original network flow threat detection method in the cyberspace scenario described in Embodiment 1 or Embodiment 2.

[0267] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0268] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0269] Finally, it should be noted that the original network flow threat detection method and device in a cyberspace scenario disclosed in the embodiment of the present invention discloses only the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that it is still possible to modify the technical solutions recorded in the aforementioned embodiments, or to replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for detecting threats in a network space scenario, characterized in that: The method comprises: S1, obtain the original network flow data; The original network flow data includes normal flow data, DDos attack network flow data, and WebAttack attack network flow data; S2, processing the original network flow data to obtain feature information of the original network flow data, including: S21, preprocessing the original network flow data to obtain standardized original network flow data; S22, sampling the standardized original network flow data to obtain N subgraphs, where N is a positive integer; S23, performing scale normalization processing on the N sub-graphs to obtain N sub-graphs of the same size; S24, performing feature extraction on the N subgraphs of the same size to obtain feature information of the original network flow data, including: S241, using a preset feature extraction model, extracting features from any sub-graph to obtain sub-graph feature information; The feature extraction model expression is: Where t represents time, ω represents angular frequency, x() represents sub-image, τ represents delay, W(t,ω) is the sub-image feature information, and * represents conjugation; S242, using a preset filtering model, filtering the sub-image feature information W(t,ω) to obtain the sub-image feature information after noise reduction and edge preservation S243, performing feature fusion on the sub-graph feature information after denoising and edge preservation of the N sub-graphs to obtain feature information of the original network flow data, including: S2431, using a VGG19 network to process the sub-image feature information after denoising and edge preservation of the N sub-images, to obtain N first sub-image feature information; S2432, clustering the N first subgraph feature information using an agglomerative hierarchical clustering method to obtain N cluster feature information; S2433, using a feature fusion model, performing feature fusion on the N cluster feature information to obtain feature information of the original network flow data; S3, dividing the feature information of the original network flow data to obtain training feature information of the original network flow data and test feature information of the original network flow data; S4, using a preset automatic machine learning model to process the training feature information of the original network flow data to obtain an optimized original network flow threat detection model; S5, using the optimized original network flow threat detection model, processing the test feature information of the original network flow data to obtain an original network flow threat detection result.

2. The original network flow threat detection method in the cyberspace scenario according to claim 1 is characterized in that: The preprocessing of the original network flow data to obtain standardized original network flow data includes: S211, performing data alignment on the original network flow data to obtain alignment data information; S212, performing internal data padding on the aligned data information to obtain padded data information; S213, performing data representation on the padded data information to obtain standardized original network flow data.

3. The original network flow threat detection method in the cyberspace scenario according to claim 1 is characterized in that: The preset filtering model expression is: Where W() represents the atomic graph feature information, represents the feature information of the sub-image after filtering, m represents the output pixel point, n represents the input pixel point, S represents the set rectangular frame area for traversing the sub-image, W is the two-dimensional template, and are two Gaussian functions, representing the spatial domain kernel and the pixel domain kernel respectively, M m The specific expression is: and The specific expression is: In the formula, a and b represent the horizontal and vertical coordinates of the input pixel, i and j represent the coordinates of the center of the box, and σ s , σ r Represents the standard deviation of the Gaussian function.

4. The original network flow threat detection method in the cyberspace scenario according to claim 1 is characterized in that: The calculation method of the feature fusion model includes: Obtain cluster feature information X and cluster feature information Y to be fused; Performing feature projection on the cluster feature information X and the cluster feature information Y to obtain projected cluster feature information X' and projected cluster feature information Y'; Processing the projected cluster feature information X' and the projected cluster feature information Y' to obtain a projection vector a1 and a projection vector b1; The cluster feature information X, the cluster feature information Y, the projection vector a1 and the projection vector b1 are processed to obtain fused feature information of the cluster feature information X and the cluster feature information Y.

5. A device for detecting threats of original network flows in a cyberspace scenario, characterized in that: The device comprises: The acquisition module is used to obtain the original network flow data; The original network flow data includes normal flow data, DDos attack network flow data, and WebAttack attack network flow data; The first processing module is used to process the original network flow data to obtain feature information of the original network flow data, including: S21, preprocessing the original network flow data to obtain standardized original network flow data; S22, sampling the standardized original network flow data to obtain N subgraphs, where N is a positive integer; S23, performing scale normalization processing on the N sub-graphs to obtain N sub-graphs of the same size; S24, performing feature extraction on the N subgraphs of the same size to obtain feature information of the original network flow data, including: S241, using a preset feature extraction model, extracting features from any sub-graph to obtain sub-graph feature information; The feature extraction model expression is: Where t represents time, ω represents angular frequency, x() represents sub-image, τ represents delay, W(t,ω) is the sub-image feature information, and * represents conjugation; S242, using a preset filtering model, filtering the sub-image feature information W(t,ω) to obtain the sub-image feature information after noise reduction and edge preservation S243, performing feature fusion on the sub-graph feature information after denoising and edge preservation of the N sub-graphs to obtain feature information of the original network flow data, including: S2431, using a VGG19 network to process the sub-image feature information after denoising and edge preservation of the N sub-images, to obtain N first sub-image feature information; S2432, clustering the N first subgraph feature information using an agglomerative hierarchical clustering method to obtain N cluster feature information; S2433, using a feature fusion model, performing feature fusion on the N cluster feature information to obtain feature information of the original network flow data; A second processing module is used to divide the characteristic information of the original network flow data to obtain training characteristic information of the original network flow data and test characteristic information of the original network flow data; A third processing module is used to process the training feature information of the original network flow data using a preset automatic machine learning model to obtain an optimized original network flow threat detection model; The fourth processing module is used to use the optimized original network flow threat detection model to process the test feature information of the original network flow data to obtain the original network flow threat detection result.

6. A device for detecting threats of original network flows in a cyberspace scenario, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the original network flow threat detection method in the cyberspace scenario as described in any one of claims 1-4.

7. A computer storable medium, characterized in that: The computer storable medium stores computer instructions, which, when called, are used to execute the original network flow threat detection method in the cyberspace scenario as described in any one of claims 1-4.

Images