Abnormal traffic detection method, device, equipment, medium and program product

Abstract

The embodiment of the invention provides an abnormal traffic detection method and device, equipment, a medium and a program product, is applied to computer equipment, and comprises the following steps: acquiring abnormal traffic data, and creating an abnormal traffic data set according to the abnormal traffic data; preprocessing the abnormal traffic data set to obtain a preprocessed data set; inputting the preprocessed data set into a word vector pre-training model to output a dynamic word vector; inputting the dynamic word vectors into a bidirectional long-short term memory network model to output global features; inputting the dynamic word vectors into a bidirectional gating loop unit model to output local features; performing feature fusion according to the global features and the local features to generate an abnormal flow detection result; and the abnormal flow detection result is output, so that the detection accuracy of the abnormal flow is improved.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to an abnormal traffic detection method, device, equipment, medium and program product. Background Art

[0002] With the development of the Internet of Things, cloud computing and big data technologies, the complexity and scale of network traffic are increasing. In order to ensure network security, abnormal traffic detection technology is used to identify abnormal activities in normal traffic and promptly discover security incidents such as network attacks.

[0003] In the prior art, methods for detecting abnormal traffic mainly include deep learning detection methods and machine learning detection methods.

[0004] However, the detection methods in the prior art rely on static rules and signature comparisons, which are unable to identify complex traffic features in abnormal traffic data. There are false alarms when facing unknown attacks or variant attacks, resulting in reduced accuracy in detecting abnormal traffic. Summary of the invention

[0005] The embodiments of the present application provide an abnormal traffic detection method, device, equipment, medium and program product to solve the problem of reduced abnormal traffic detection accuracy in the prior art.

[0006] In a first aspect, an embodiment of the present application provides an abnormal traffic detection method, which is applied to a computer device, comprising:

[0007] Acquire abnormal traffic data, and create an abnormal traffic data set according to the abnormal traffic data;

[0008] Preprocessing the abnormal traffic data set to obtain a preprocessed data set;

[0009] Inputting the preprocessed data set into a word vector pre-training model to output a dynamic word vector;

[0010] Inputting the dynamic word vector into a bidirectional long short-term memory network model to output global features;

[0011] Inputting the dynamic word vector into a bidirectional gated recurrent unit model to output local features;

[0012] Perform feature fusion according to the global features and the local features to generate an abnormal traffic detection result;

[0013] The abnormal traffic detection result is output.

[0014] In a possible implementation, the bidirectional long short-term memory network model includes a forward long short-term memory network model and a backward long short-term memory network model; the step of inputting the dynamic word vector into the bidirectional long short-term memory network model to output global features includes: inputting the dynamic word vector into the forward long short-term memory network model to output a forward long short-term memory sequence; inputting the dynamic word vector into the backward long short-term memory network model to output a backward long short-term memory sequence; concatenating the forward long short-term memory sequence and the backward long short-term memory sequence to obtain a long short-term memory output sequence; and inputting the long short-term memory output sequence into a global attention layer to output global features.

[0015] In one possible implementation, the bidirectional gated recurrent unit model includes a forward gated recurrent unit model and a backward gated recurrent unit model; the step of inputting the dynamic word vector into the bidirectional gated recurrent unit model to output local features includes: inputting the dynamic word vector into the forward gated recurrent unit model to output a forward gated recurrent unit sequence; inputting the dynamic word vector into the backward gated recurrent unit model to output a backward gated recurrent unit sequence; concatenating the forward gated recurrent unit sequence and the backward gated recurrent unit sequence to obtain a gated recurrent unit sequence; and inputting the gated recurrent unit sequence into a local attention layer to output local features.

[0016] In a possible implementation, the feature fusion based on the global features and the local features to generate an abnormal traffic detection result includes: generating a global feature matrix based on the global features; generating a local feature matrix based on the local features; performing matrix concatenation of the global feature matrix and the local feature matrix to obtain a feature vector; and inputting the feature vector into a classifier to output an abnormal traffic detection result.

[0017] In a possible implementation manner, the model for inputting the feature vector into a classifier to output an abnormal traffic detection result is:

[0018] p=softmax(w0V * +b0)

[0019] Where p represents the predicted probability of abnormal traffic; w0 represents the weight matrix; V * represents the feature vector; b0 represents the offset.

[0020] In a possible implementation, preprocessing the abnormal traffic data set to obtain a preprocessed data set includes: performing IP grouping processing on the abnormal traffic data set according to a nested dictionary to obtain a grouped data set; obtaining derived feature fields in the grouped data set; and splicing the derived feature fields according to a preset field length to obtain the preprocessed data set.

[0021] In a second aspect, an embodiment of the present application provides an abnormal traffic detection device, which is applied to a computer device, including:

[0022] An acquisition module, used for acquiring abnormal traffic data and creating an abnormal traffic data set according to the abnormal traffic data;

[0023] A preprocessing module, used for preprocessing the abnormal traffic data set to obtain a preprocessed data set;

[0024] A first output module, used for inputting the preprocessed data set into a word vector pre-training model to output a dynamic word vector;

[0025] A second output module is used to input the dynamic word vector into a bidirectional long short-term memory network model to output global features;

[0026] A third output module, used for inputting the dynamic word vector into a bidirectional gated recurrent unit model to output local features;

[0027] A fusion module, used for performing feature fusion according to the global feature and the local feature to generate an abnormal traffic detection result;

[0028] The fourth output module is used to output the abnormal flow detection result.

[0029] In a third aspect, an embodiment of the present application provides an abnormal flow detection device, including:

[0030] at least one processor and memory;

[0031] The memory stores computer-executable instructions;

[0032] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the above first aspect and / or various possible implementations of the first aspect.

[0033] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0034] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0035] The abnormal traffic detection method, device, equipment, medium and program product provided in the embodiments of the present application perform text vectorization on the preprocessed data set through a word vector pre-training model to obtain dynamic word vectors in the data traffic, process the word vectors through a bidirectional long short-term memory network model and a bidirectional gated recurrent unit model, output global features and local features, and perform feature fusion on the global features and local features to generate abnormal traffic detection results. Compared with the prior art, dynamic word vectors are obtained by constructing a word vector pre-training model, which avoids the problem of the prior art relying on static rules and signature comparison; dynamic word vectors are processed by using a bidirectional long short-term memory network model and a gated recurrent unit model, which avoids the problem of unidirectional text reading of the traditional long short-term memory network model, and improves the detection accuracy of abnormal traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0037] Figure 1 A schematic diagram of the system structure of a computer device provided in an embodiment of the present application;

[0038] Figure 2 A flow chart of the abnormal traffic detection method provided in this application;

[0039] Figure 3 A schematic diagram of the structure of the Bert model provided in the embodiment of the present application;

[0040] Figure 4 A schematic diagram of the structure of a bidirectional long short-term memory network model provided in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of the structure of a bidirectional gated recurrent unit model provided in an embodiment of the present application;

[0042] Figure 6 A schematic diagram of the structure of a long short-term memory network neuron provided in an embodiment of the present application;

[0043] Figure 7 A schematic diagram of the structure of a gated recurrent unit neuron provided in an embodiment of the present application;

[0044] Figure 8 A schematic diagram of the structure of the abnormal flow detection device provided in this application;

[0045] Fig. 9 This is a schematic diagram of the structure of the abnormal flow detection device provided in this application.

[0046] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0048] First, the terms involved in this application are explained:

[0049] Honeypot technology: Honeypot technology is an effective means to obtain network traffic data sets. By building bait hosts, network services or applications, honeypots can attract attackers to attack, thereby capturing the attacker's activities and attack methods. When deploying a honeypot, one or more virtual or physical hosts are usually configured and vulnerable services are simulated, which may become targets in a real environment. During the interaction between the attacker and the honeypot, the system will record all network traffic and generate detailed logs. The logs and captured network data packets are usually saved in .pcap files. Through the data files, technicians can identify the tools and techniques used by attackers, assess the threat level of the attack, and develop defense strategies.

[0050] Fwd Header Len: indicates the total number of bytes in the forward data packet header. It is obtained by accessing the header length of each IP packet.

[0051] Down / Up Ratio: Indicates the ratio of download and upload. The ratio is obtained by calculating the number of forward and reverse packets.

[0052] Fwd IAT Mean: indicates the average value of forward IAT (Inter-Arrival Time), which is obtained by calculating the time interval between forward traffic packets.

[0053] Flow IAT Mean: Indicates the average IAT value of all packets, obtained by calculating the time interval between all packets.

[0054] Flow Duration: Indicates the duration of the flow, which is obtained by calculating the time difference between the first data packet and the last data packet.

[0055] Active Max: indicates the maximum active time. The maximum active time is obtained by calculating the time difference between the idle time and the active time.

[0056] Active Std: Indicates the standard deviation of the active time.

[0057] Fwd IAT Max: Indicates the maximum value of forward IAT.

[0058] Bwd IAT Min: Indicates the minimum value of the reverse IAT. It is obtained by calculating the time interval between reverse traffic packets and taking the minimum value.

[0059] Tot Fwd Pkts: indicates the total number of forward packets. It is obtained by obtaining the number of forward packets.

[0060] Idle Mean: indicates the average value of the previous idle time. It is obtained by calculating the average of all idle time.

[0061] Active Mean: Indicates the average value of the previous activity time. It is obtained by calculating the average value of the activity time.

[0062] Bwd Pkts / s: indicates the number of reverse packets per second. It is obtained by dividing the number of reverse packets by the duration of the flow (in seconds).

[0063] Fwd Pkt Len Std: Indicates the standard deviation of the forward packet length.

[0064] Fwd Act Data Pkts: Indicates packets in the forward direction with a payload of at least one byte of TCP data.

[0065] Pkt Size Avg: Indicates the average size of data packets.

[0066] Idle Max: indicates the maximum idle time. It is obtained by taking the maximum value of all idle times.

[0067] Active Min: Indicates the minimum active time. It is obtained by taking the minimum value of the active time.

[0068] ACK Flag Count: Indicates the number of packets with ACK flag observed.

[0069] Bwd Seg Size Avg: Indicates the average size of packets in the reverse direction.

[0070] Bert model: It is a word vector pre-training model based on the bidirectional Transformer encoder structure, including the masked language model and the next sentence prediction. The Bert model extracts deep information about the text dynamics and combines the masked language model with the next sentence prediction to achieve text vectorization and extraction of semantic information.

[0071] With the development of the Internet of Things, cloud computing and big data technologies, the complexity and scale of network traffic are increasing. In order to ensure network security, abnormal traffic detection technology is used to identify abnormal activities in normal traffic and timely discover security incidents such as network attacks. In the prior art, the methods for detecting abnormal traffic mainly include deep learning detection methods and machine learning detection methods. However, the detection methods of the prior art rely on static rules and signature comparisons, and are unable to identify complex traffic features in abnormal traffic data. There are false alarms when facing unknown attacks or variant attacks, resulting in reduced accuracy in detecting abnormal traffic.

[0072] In order to solve the above technical problems, the embodiments of the present application propose the following technical concepts: the inventors consider obtaining abnormal traffic data, pre-processing the abnormal traffic data and inputting it into the word vector pre-training model, performing text vectorization, and obtaining dynamic word vectors, and consider designing a bidirectional long short-term memory network model and a bidirectional gated recurrent unit model to process word vectors, obtain global features and local features, perform feature fusion on the global features and local features, and generate abnormal traffic detection results. The following is a detailed description using a detailed embodiment.

[0073] Figure 1 A schematic diagram of the system structure of a computer device provided in an embodiment of the present application. Figure 1 As shown, the computer device includes: a receiving device 101, a processor 102 and a display device 103.

[0074] It is understandable that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the object identification method. In other feasible implementations of the present application, the above architecture may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0075] In a specific implementation process, the receiving device 101 may be an input / output interface or a communication interface, and may obtain abnormal traffic data.

[0076] The processor 102 may generate abnormal traffic detection results.

[0077] The display device 103 can be used to display the above abnormal flow detection results and the like.

[0078] The display device may also be a touch display screen, which is used to receive user instructions while displaying the above-mentioned content, so as to realize operational interaction with the user.

[0079] It should be understood that the above-mentioned processor can be implemented by the processor reading instructions in the memory and executing the instructions, or it can be implemented by a chip circuit.

[0080] In addition, the network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0081] Figure 2 This is a flow chart of the abnormal traffic detection method provided by this application. The execution subject of this embodiment can be a computer device, and this embodiment is not particularly limited here. Figure 2 As shown, the method includes:

[0082] S201: Acquire abnormal traffic data, and create an abnormal traffic data set according to the abnormal traffic data.

[0083] In this embodiment, the abnormal traffic data is obtained by using the honeypot technology, and an abnormal traffic data set is created according to the abnormal traffic data.

[0084] Specifically, a bait host, network service or application is constructed to capture the attacker's data packet file, and the data packet file is parsed to obtain an abnormal traffic data set.

[0085] In this embodiment, the data packet file includes but is not limited to the number of data packets, data packet size, timestamp, source IP address, destination IP address, IP protocol type, protocol port, TCP / UDP sequence number, flag, flow direction, flow duration, flow ratio, interval time (IAT), rates of different flows, and packet loss and delay information.

[0086] S202: Preprocess the abnormal traffic data set to obtain a preprocessed data set.

[0087] Specifically, the data in the abnormal traffic data set is grouped according to the source IP and destination IP, and the derived features are calculated based on the representative fields to obtain the derived feature fields. The derived fields are spliced ​​according to the preset field length to obtain the preprocessed data set.

[0088] S203: Input the preprocessed data set into the word vector pre-training model to output a dynamic word vector.

[0089] In this embodiment, the word vector pre-training model is a Bert model.

[0090] The input sequence of the Bert model consists of position embedding, word embedding, and fragment embedding. The input is the data extracted from each group of data packets in the abnormal traffic dataset, and the output is the word vector calculated after the field of the flow data is integrated with the global semantic information.

[0091] Figure 3 A schematic diagram of the structure of the Bert model provided in the embodiment of the present application.

[0092] like Figure 3 As shown, x1,x2,…,x n is the output word vector sequence of the Bert model.

[0093] S204: Input the dynamic word vector into the bidirectional long short-term memory network model to output global features.

[0094] Figure 4 A schematic diagram of the structure of a bidirectional long short-term memory network model provided in an embodiment of the present application.

[0095] Specifically, the dynamic word vector is input into the bidirectional long short-term memory network model to obtain a forward long short-term memory sequence and a backward long short-term memory sequence, the forward long short-term memory sequence and the backward long short-term memory sequence are spliced, and the spliced ​​sequence is input into the attention layer, and the global features are output through the global attention mechanism.

[0096] S205: Input the dynamic word vector into the bidirectional gated recurrent unit model to output local features.

[0097] Figure 5 A schematic diagram of the structure of a bidirectional gated recurrent unit model provided in an embodiment of the present application.

[0098] Specifically, the dynamic word vector is input into the bidirectional gated recurrent unit model to obtain a forward gated recurrent unit sequence and a backward gated recurrent unit sequence, the forward gated recurrent unit sequence and the backward gated recurrent unit sequence are concatenated, and the concatenated sequence is input into the attention layer, and the local features are output through the local attention mechanism.

[0099] S206: Perform feature fusion based on global features and local features to generate abnormal traffic detection results.

[0100] Specifically, the global features and local features optimized by the attention mechanism are converted into matrix form to obtain a global feature matrix and a local feature matrix. The row vectors of the global feature matrix and the local feature matrix are concatenated to obtain a concatenated matrix. The concatenated matrix is ​​input into the softmax classifier to generate abnormal traffic detection results.

[0101] In this embodiment, the abnormal traffic detection result is the predicted probability of abnormal traffic, which is between 0 and 1, where 0 represents abnormal traffic and 1 represents normal traffic.

[0102] S207: Output abnormal traffic detection results.

[0103] Specifically, the model accuracy of the abnormal traffic detection model is adjusted according to the output abnormal traffic detection result, and the traffic data to be detected is input into the adjusted abnormal traffic detection model to detect abnormal traffic.

[0104] It can be seen from the above embodiments that the preprocessed data set is text-vectorized through a word vector pre-training model to obtain dynamic word vectors in the data flow, and the word vectors are processed through a bidirectional long short-term memory network model and a bidirectional gated recurrent unit model to output global features and local features, and feature fusion is performed on the global features and local features to generate abnormal traffic detection results. Compared with the prior art, dynamic word vectors are obtained by constructing a word vector pre-training model, which avoids the problem of the prior art relying on static rules and signature comparison; dynamic word vectors are processed using a bidirectional long short-term memory network model and a gated recurrent unit model, which avoids the problem of unidirectional reading of text by the traditional long short-term memory network model, and improves the detection accuracy of abnormal traffic.

[0105] In one embodiment of the present application, the bidirectional long short-term memory network model includes a forward long short-term memory network model and a backward long short-term memory network model; step S204 includes:

[0106] S2041: Input the dynamic word vector into the forward long short-term memory network model to output a forward long short-term memory sequence.

[0107] Figure 6 A schematic diagram of the structure of a long short-term memory network neuron provided in an embodiment of the present application.

[0108] like Figure 6 As shown, the long short-term memory network includes a forget gate, a memory gate, and an output gate.

[0109] Specifically, the calculation formula of the forget gate is:

[0110] f t =σ(W f ×[h t-1 ,x t]+b f )

[0111] In the formula, f t represents the output value of the forget gate; σ represents the sigmoid function; W f represents the weight matrix of the forget gate; b f represents the offset vector of the forget gate; h t-1 represents the hidden state at the previous moment; x t Indicates the input word at the current moment.

[0112] Specifically, the calculation formula of the memory gate is:

[0113] i t =σ(W i ×[h t-1 ,x t ]+b i )

[0114]

[0115] In the formula, i t represents the output value of the memory gate; σ represents the sigmoid function; W i represents the weight matrix of the memory gate; b i represents the offset vector of the memory gate; h t-1 represents the hidden state at the previous moment; x t Represents the input word at the current moment; Indicates temporary cell state; W c b represents the weight matrix of the temporary cell state; b c An offset vector representing the temporary cell state.

[0116] Specifically, the calculation formula of the cell state at the current moment is:

[0117]

[0118] In the formula, C t Indicates the current cell state; C t-1 Indicates the cell state at the previous moment; f t Represents the output value of the forget gate; i t Represents the output value of the memory gate; Indicates a temporary cell state.

[0119] Specifically, the calculation formula for the output gate and the hidden state at the current moment is:

[0120] o t =σ(W o ×[h t-1 ,x t ]+b o)

[0121] h t =o t *tanh(C t )

[0122] In the formula, o t represents the output value of the output gate; σ represents the sigmoid function; W o represents the weight matrix of the output gate; b o represents the offset vector of the output gate; h t Indicates the hidden state at the current moment.

[0123] In this embodiment, after training the forward sequence, the forward long short-term memory sequence h is obtained. L .

[0124] in,

[0125] S2042: Input the dynamic word vector into the backward long short-term memory network model to output a backward long short-term memory sequence.

[0126] In this embodiment, after the backward sequence training, the backward long short-term memory sequence h is obtained. R .

[0127] in,

[0128] S2043: Concatenate the forward long short-term memory sequence and the backward long short-term memory sequence to obtain a long short-term memory output sequence.

[0129] Specifically, the forward long short-term memory sequence and the backward long short-term memory sequence are concatenated to obtain a long short-term memory output sequence.

[0130] Among them, the spliced ​​long short-term memory output sequence is recorded as h ′ t .

[0131] S2044: Input the long short-term memory output sequence into the global attention layer to output global features.

[0132] Specifically, the LSTM output sequence is input into the attention layer, and weights are assigned to the fields through the global attention mechanism to output global features.

[0133] Among them, the calculation formula of the global attention layer includes:

[0134] u t =tanh(w s h ′ t +b s )

[0135]

[0136] In the formula, w s Represents the weight matrix of the global attention mechanism of the attention layer; b s h represents the offset vector of the global attention mechanism of the attention layer; ′ t represents the concatenated long short-term memory output sequence; u t Indicates the correlation between each element in the spliced ​​long short-term memory output sequence and the spliced ​​sequence; u s represents an initial training parameter; α t Represents the attention score of the global feature; V represents the feature vector output by the global attention mechanism, that is, the global feature.

[0137] From the above embodiments, it can be seen that by creating a bidirectional LSTM network model, the forward LSTM network model processes the forward text in the word vector and trains the forward sequence; the backward LSTM network model processes the backward text in the word vector and trains the backward sequence, and the bidirectional LSTM network model extracts global feature information, thereby avoiding the problem that the traditional LSTM network model only reads text in a single direction.

[0138] In one embodiment of the present application, the bidirectional gated recurrent unit model includes a forward gated recurrent unit model and a backward gated recurrent unit model; step S205 includes:

[0139] S2051: Input the dynamic word vector into the forward gated recurrent unit model to output a forward gated recurrent unit sequence.

[0140] Figure 7 A schematic diagram of the structure of a gated recurrent unit neuron provided in an embodiment of the present application.

[0141] like Figure 7 As shown, the gated recurrent unit includes a reset gate and an update gate.

[0142] Specifically, the calculation formula in the gated recurrent unit includes:

[0143] r t =σ(w r ·[h t-1 ,x t ])

[0144] z t =σ(w z ·[h t-1 ,x t ])

[0145]

[0146] h t =(1-z t )*h t-1 +z t *h t

[0147] In the formula, r t represents the output of the reset gate at time t; z t represents the output of the update gate at time t; h t-1 Represents the hidden layer state at time t-1; represents the candidate activation state at time t; h t represents the activation state at time t; w r represents the weight matrix of the reset gate; w z represents the weight matrix of the update gate; w represents the weight matrix of the candidate activation state; σ represents the sigmoid activation function; x t Indicates the input word at the current moment.

[0148] In this embodiment, the forward gated recurrent unit sequence output by the forward gated recurrent unit model is expressed as:

[0149]

[0150] Where T represents the length of the time series.

[0151] S2052: Input the dynamic word vector into the backward gated recurrent unit model to output a backward gated recurrent unit sequence.

[0152] In this embodiment, the backward gated recurrent unit sequence output by the backward gated recurrent unit model is expressed as:

[0153]

[0154] Where T represents the length of the time series.

[0155] S2053: Concatenate the forward gated recurrent unit sequence and the backward gated recurrent unit sequence to obtain a gated recurrent unit sequence.

[0156] In this embodiment, the spliced ​​gated recurrent unit sequence is expressed as:

[0157]

[0158] S2054: Input the gated recurrent unit sequence into the local attention layer to output local features.

[0159] Specifically, the gated recurrent unit sequence is input into the attention layer, and weights are assigned through the local attention mechanism to output local features.

[0160] Among them, the calculation formula of the local attention layer includes:

[0161] u t ′ =tanh(w w h ′ t ′ +b w )

[0162]

[0163] In the formula, w w Represents the weight matrix of the local attention mechanism of the attention layer; b w h represents the offset vector of the local attention mechanism of the attention layer; ′ t ′ represents the spliced ​​gated recurrent unit sequence; u t ′ Indicates the correlation between each element in the spliced ​​gated recurrent unit sequence and the spliced ​​sequence; u w represents an initial training parameter; α t ′ Represents the attention score of local features; V ′ Represents the feature vector output by the local attention mechanism, that is, the local feature.

[0164] From the above embodiments, it can be seen that by creating a bidirectional gated recurrent unit model, the forward gated recurrent unit model processes the forward text in the word vector and trains the forward sequence; the backward text in the word vector is processed by the backward long short-term memory network model, and the backward sequence is trained. The local feature information is extracted by the bidirectional long short-term memory network model, which avoids the problem of long-term dependence in the long short-term memory network model and improves the efficiency of calculation.

[0165] In one embodiment of the present application, step S206 includes:

[0166] S2061: Generate a global feature matrix based on the global features.

[0167] In this embodiment, the global feature matrix is ​​denoted as V s .

[0168] S2062: Generate a local feature matrix based on the local features.

[0169] In this embodiment, the local feature matrix is ​​denoted as V e .

[0170] S2063: Concatenate the global feature matrix and the local feature matrix to obtain a feature vector.

[0171] Specifically, the row vectors of the global feature matrix and the local feature matrix are concatenated, and the concatenated matrix is ​​obtained after feature fusion, that is, the feature vector V * .

[0172] In this embodiment, the feature vector V * for (r s +r e )×c size matrix.

[0173] Among them, r s Represents the global characteristic moment V s The number of matrix rows, r e Represents the local feature V e The number of rows of the matrix, and c represents the number of columns of the matrix.

[0174] S2064: Input the feature vector into the classifier to output abnormal traffic detection results.

[0175] In one embodiment of the present application, the feature vector is input into a classifier to output a model of abnormal traffic detection results, which is:

[0176] p=softmax(w0V * +b0)

[0177] Where p represents the predicted probability of abnormal traffic; w0 represents the weight matrix; V * represents the feature vector; b0 represents the offset.

[0178] It can be seen from the above embodiments that the feature matrix of the dual-channel model is spliced ​​in the form of matrix splicing, and the feature vector is screened and classified by a classifier to generate an abnormal flow detection result, thereby improving the computing power of classifying abnormal flow.

[0179] In one embodiment of the present application, step S202 includes:

[0180] S2021: Perform IP grouping processing on the abnormal traffic data set according to the nested dictionary to obtain a grouped data set.

[0181] Exemplarily, the .pacp file is parsed by the Scapy library, and grouping is performed according to the source IP and destination IP in the data set to obtain a grouped data set.

[0182] S2022: Obtain derived feature fields in the grouped data set.

[0183] Specifically, representative fields in the grouped data set are obtained, and derived features are calculated on the representative fields to obtain derived feature fields.

[0184] In this embodiment, the derived feature fields include but are not limited to Fwd Header Len, Down / Up Ratio, FwdIAT Mean, Flow IAT Mean, Flow Duration, Active Max, Active Std, Fwd IAT Max, BwdIAT Min, Tot Fwd Pkts, Idle Mean, Active Mean, Bwd Pkts / s, Fwd Pkt Len Std, Fwd ActData Pkts, Pkt Size Avg, Idle Max, Active Min, ACK Flag Count and Bwd Seg Size Avg.

[0185] In this embodiment, representative fields include, but are not limited to, source IP, destination IP, source port number, destination port number, flow ID, and timestamp.

[0186] In this embodiment, the representative fields do not participate in model training and are only used for derived feature calculation.

[0187] S2023: splicing the derived feature fields according to a preset field length to obtain a preprocessed data set.

[0188] Specifically, if the length of the concatenated field is less than a preset length, a zero-padding operation is used to fill the field.

[0189] It can be seen from the above embodiments that by processing the abnormal traffic data according to IP groups, obtaining the derived characteristic fields in the group data set, and splicing the derived characteristic fields according to the preset field length, there is no need to process all the data in the group data set, thereby improving the efficiency of abnormal traffic data processing.

[0190] Figure 8 The schematic diagram of the structure of the abnormal flow detection device provided in this application is as follows: Figure 8 As shown, the abnormal traffic detection device 80 provided in this embodiment includes: an acquisition module 801, a preprocessing module 802, a first output module 803, a second output module 804, a third output module 805, a fusion module 806 and a fourth output module 807.

[0191] The acquisition module 801 is used to acquire abnormal traffic data and create an abnormal traffic data set according to the abnormal traffic data.

[0192] The preprocessing module 802 is used to preprocess the abnormal traffic data set to obtain a preprocessed data set.

[0193] The first output module 803 is used to input the preprocessed data set into the word vector pre-training model to output a dynamic word vector.

[0194] The second output module 804 is used to input the dynamic word vector into the bidirectional long short-term memory network model to output the global feature.

[0195] The third output module 805 is used to input the dynamic word vector into the bidirectional gated recurrent unit model to output local features.

[0196] The fusion module 806 is used to perform feature fusion according to the global features and the local features to generate abnormal traffic detection results.

[0197] The fourth output module 807 is used to output abnormal traffic detection results.

[0198] In a possible implementation, the second output module 804 includes:

[0199] The first output unit 8041 is used to input the dynamic word vector into the forward long short-term memory network model to output a forward long short-term memory sequence.

[0200] The second output unit 8042 is used to input the dynamic word vector into the backward long short-term memory network model to output a backward long short-term memory sequence.

[0201] The first concatenation unit 8043 is used to concatenate the forward long short-term memory sequence and the backward long short-term memory sequence to obtain a long short-term memory output sequence.

[0202] The third output unit 8044 is used to input the long short-term memory output sequence into the global attention layer to output the global feature.

[0203] In a possible implementation, the third output module 805 includes:

[0204] The fourth output unit 8051 is used to input the dynamic word vector into the forward gated recurrent unit model to output a forward gated recurrent unit sequence.

[0205] The fifth output unit 8052 is used to input the dynamic word vector into the backward gated recurrent unit model to output a backward gated recurrent unit sequence.

[0206] The second concatenation unit 8053 is used to concatenate the forward gated recurrent unit sequence and the backward gated recurrent unit sequence to obtain a gated recurrent unit sequence.

[0207] The sixth output unit 8054 is used to input the gated recurrent unit sequence into the local attention layer to output local features.

[0208] In a possible implementation, the fusion module 806 includes:

[0209] The first generating unit 8061 is used to generate a global feature matrix according to the global features.

[0210] The second generating unit 8062 is used to generate a local feature matrix according to the local features.

[0211] The third concatenation unit 8063 is used to concatenate the global feature matrix and the local feature matrix to obtain a feature vector.

[0212] The seventh output unit 8064 is used to input the feature vector into the classifier to output the abnormal traffic detection result.

[0213] In a possible implementation manner, in the seventh output unit 8064, the model for outputting the abnormal traffic detection result is:

[0214] p=softmax(w0V * +b0)

[0215] Where p represents the predicted probability of abnormal traffic; w0 represents the weight matrix; V * represents the feature vector; b0 represents the offset.

[0216] In a possible implementation, the preprocessing module 802 includes:

[0217] The grouping unit 8021 is used to perform IP grouping processing on the abnormal traffic data set according to the nested dictionary to obtain a grouped data set.

[0218] The acquisition unit 8022 is used to acquire the derived feature fields in the grouped data set.

[0219] The fourth concatenation unit 8023 is used to concatenate the derived characteristic fields according to a preset field length to obtain a preprocessed data set.

[0220] The database selection delivery processing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0221] Fig. 9 This is a schematic diagram of the structure of the abnormal flow detection device provided in this application. Fig. 9As shown, the abnormal traffic detection device 90 provided in this embodiment includes: at least one processor 901 and a memory 902. Optionally, the device 90 also includes a communication component 903. The processor 901, the memory 902 and the communication component 903 are connected via a bus 904.

[0222] In a specific implementation process, at least one processor 901 executes the computer execution instructions stored in the memory 902, so that at least one processor 901 executes the above-mentioned abnormal traffic detection method.

[0223] The specific implementation process of the processor 901 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0224] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0225] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0226] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0227] The present application also provides a computer program product, including a computer program, which implements the above-mentioned abnormal traffic detection method when executed by a processor.

[0228] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned abnormal traffic detection method is implemented.

[0229] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0230] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0231] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0232] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0233] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0234] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0235] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0236] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for detecting abnormal traffic, characterized in that: Applicable to computer equipment, including: Acquire abnormal traffic data, and create an abnormal traffic data set according to the abnormal traffic data; Preprocessing the abnormal traffic data set to obtain a preprocessed data set; Inputting the preprocessed data set into a word vector pre-training model to output a dynamic word vector; Inputting the dynamic word vector into a bidirectional long short-term memory network model to output global features; Inputting the dynamic word vector into a bidirectional gated recurrent unit model to output local features; Perform feature fusion according to the global features and the local features to generate an abnormal traffic detection result; The abnormal traffic detection result is output.

2. The method according to claim 1, characterized in that The bidirectional long short-term memory network model includes a forward long short-term memory network model and a backward long short-term memory network model; the step of inputting the dynamic word vector into the bidirectional long short-term memory network model to output global features includes: Inputting the dynamic word vector into the forward long short-term memory network model to output a forward long short-term memory sequence; Inputting the dynamic word vector into the backward long short-term memory network model to output a backward long short-term memory sequence; Concatenate the forward long short-term memory sequence and the backward long short-term memory sequence to obtain a long short-term memory output sequence; The LSTM output sequence is input into a global attention layer to output global features.

3. The method according to claim 1, characterized in that The bidirectional gated recurrent unit model includes a forward gated recurrent unit model and a backward gated recurrent unit model; the step of inputting the dynamic word vector into the bidirectional gated recurrent unit model to output local features includes: Inputting the dynamic word vector into the forward gated recurrent unit model to output a forward gated recurrent unit sequence; Inputting the dynamic word vector into the backward gated recurrent unit model to output a backward gated recurrent unit sequence; splicing the forward gated recurrent unit sequence and the backward gated recurrent unit sequence to obtain a gated recurrent unit sequence; The gated recurrent unit sequence is input into a local attention layer to output local features.

4. The method according to claim 1, characterized in that: The step of fusing features according to the global features and the local features to generate abnormal traffic detection results includes: Generate a global feature matrix according to the global features; Generate a local feature matrix according to the local features; Perform matrix concatenation of the global feature matrix and the local feature matrix to obtain a feature vector; The feature vector is input into a classifier to output an abnormal traffic detection result.

5. The method according to claim 4, characterized in that The model for inputting the feature vector into a classifier to output abnormal traffic detection results is: p=softmax(w0V * +b0) Where p represents the predicted probability of abnormal traffic; w0 represents the weight matrix; V * represents the feature vector; b0 represents the offset.

6. The method according to any one of claims 1 to 5, characterized in that: The preprocessing of the abnormal traffic data set to obtain a preprocessed data set includes: Perform IP grouping processing on the abnormal traffic data set according to the nested dictionary to obtain a grouped data set; Obtaining derived feature fields in the grouped data set; The derived feature fields are concatenated according to a preset field length to obtain a preprocessed data set.

7. An abnormal flow detection device, characterized in that: Applicable to computer equipment, including: An acquisition module, used for acquiring abnormal traffic data and creating an abnormal traffic data set according to the abnormal traffic data; A preprocessing module, used for preprocessing the abnormal traffic data set to obtain a preprocessed data set; A first output module, used for inputting the preprocessed data set into a word vector pre-training model to output a dynamic word vector; A second output module is used to input the dynamic word vector into a bidirectional long short-term memory network model to output global features; A third output module, used for inputting the dynamic word vector into a bidirectional gated recurrent unit model to output local features; A fusion module, used for performing feature fusion according to the global feature and the local feature to generate an abnormal traffic detection result; The fourth output module is used to output the abnormal flow detection result.

8. An abnormal flow detection device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the abnormal traffic detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the abnormal traffic detection method according to any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the abnormal traffic detection method according to any one of claims 1 to 6.

Images