A web attack detection method based on a gated Transformer

Abstract

The application provides a Web attack detection method based on a gated Transformer, and relates to the technical field of network maintenance.The method provided by the application proposes a network model based on a gated Transformer, which combines a Transformer and a gated convolution module together, the Transformer extracts global semantic information of different spatial dimensions through a multi-head self-attention mechanism, the gated convolution extracts information of a local space through a one-dimensional convolution kernel, and a gating mechanism is adopted to screen and filter text information.The application can effectively extract multidimensional global features and local features, a mixed word vector table can contain more accurate and rich semantic information, can automatically extract effective data information features in a text sequence, does not need to manually screen information and replace a word table, further improves the accuracy of model multi-classification attack detection and reduces a false positive rate, and can fully protect the security of a Web server system.

Description

Technical Field

[0001] This invention relates to the field of network maintenance technology, and more specifically to a Web attack detection method based on gated Transformer. Background Technology

[0002] With the rapid development of science and technology, web applications are undoubtedly the best channel for obtaining information from devices such as computers and mobile phones, and browsers are the most frequently used application by people in all walks of life almost every day. While bringing convenience, this has also led to various security problems. Attack methods targeting web applications are constantly evolving, and once an attack is successful, it can directly threaten our daily applications and even cause privacy and security issues such as information leakage. Without protection, this can cause very serious losses. Common injection attacks include SQL injection, XSS attacks, and command injection attacks.

[0003] With the continuous development of machine learning and big data analytics, deep learning technology has been gradually applied to the field of attack detection. However, some existing methods have many shortcomings. For example, some deep learning models, such as CNNs, are strong in local sequence feature extraction but weak in global information perception and text sequence information modeling. Models like LSTM and RNN also perform poorly in long-distance global dependency features, while HTTP messages are lengthy and complex. In the field of attack detection, models need to output results as quickly as possible. The Transformer model overcomes the limitation of LSTM and RNN models in parallel computation, enabling results to be obtained in a shorter time. For relatively long HTTP messages, which contain many symbols, numbers, and other information that do not contain any information, how to automatically filter and extract key and effective information from complex information while ignoring irrelevant information is also a crucial issue for further improving the accuracy of attack detection.

[0004] Chinese patent "CN113691542A Web Attack Detection Method and Related Device Based on HTTP Request Text" provides an attack detection method based on the BiLSTM attention mechanism, which includes an expert dictionary and special dictionary replacement to reduce the dictionary space, ultimately performing multi-category attack classification. However, the multi-head attention mechanism BiLSTM model in this patent performs poorly in terms of long-distance global dependency features and cannot overcome the limitations of parallel computation. HTTP message text is lengthy and complex, containing much invalid information such as symbols and numbers. Methods such as various dictionary replacements and regular expression matching for filtering invalid information rely on expert rule bases and replacement dictionaries, and cannot truly extract, filter, and select valid information at the semantic level. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a Web attack detection method based on gated Transformer to effectively protect the security of Web server systems, addressing the shortcomings of the prior art.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] A web attack detection method based on gated Transformer includes the following steps:

[0008] Step 1: Collect traffic using the sniff module of the Python scapy library, obtain a pcap traffic file, and extract application layer data from it;

[0009] Step 2: Perform URL decoding on the message text, and segment the text information of the URL, parameter list, user-agent, cookie, and referer fields using predefined special characters.

[0010] Step 3: The hybrid word embedding module enhances the robustness of the vector representation by fusing word embedding tables generated in two different ways. The two word embedding tables in the hybrid module are a Word2Vec word embedding table based on the Continuous Bag-of-Words (Cbow) model and a word embedding table based on the embedding layer. The word embedding table based on the embedding layer is initialized using the Xavier_uniform distribution and is continuously updated with distributed word vector representations during training. The Word2Vec word embedding table based on the Cbow model needs to be generated before the model enters the training phase. The word embedding table based on the embedding layer and the Word2Vec word embedding table based on the Cbow model map word vectors to different discrete spaces, thus achieving distributed vector representation of words.

[0011] Step 4: The text information of the HTTP message is processed by the hybrid word embedding module and converted into a series of word vectors; the series of word vectors are then input into the Transformer Encoder model for global attention feature extraction; the Transformer Encoder model consists of three parts: a position encoding module, a multi-head self-attention module, and a residual layer normalization module; the Transformer Encoder model first adds position encoding information to the vectors processed by the hybrid word embedding module, and then performs multi-dimensional sequence feature extraction through the multi-head self-attention mechanism module, and finally inputs it into the feedforward neural network module;

[0012] Step 5: Input the output of the Transformer Encoder model into the gated convolutional model. The gated convolutional model extracts local features from the data within the local receptive field. The gated convolutional model dynamically filters and selects non-critical data.

[0013] Step 6: Classify the output of the gated convolution model through the final classifier module; the softmax function converts it into a probability distribution, each dimension of the probability distribution corresponds to an attack category, and the attack category corresponding to the index of the maximum probability in the probability distribution is the final attack detection classification result.

[0014] Furthermore, the specific method of step 1 is as follows:

[0015] Step 1.1: Start the sniff network interface sniffing module of the Python scapy library to collect network traffic and save it as a pcap file;

[0016] Step 1.2: Use the rd_pcap module of the Python scapy library to read and analyze the collected pcap files, extract application layer data from the pcap files and perform data text analysis, including text information of URL, parameter list, user-agent, cookie, and referer fields;

[0017] Furthermore, in the word segmentation process in step 2, since the number of words in the input sequence that the Transformer Encoder model can process is limited, there is an upper limit to the number of words after text segmentation. This is addressed by setting a maximum sentence length. Words exceeding the maximum sentence length will be removed. If the number of words in a sentence is less than the maximum length, padding will be used to fill the gaps.

[0018] Furthermore, the specific method of step 3 is as follows:

[0019] Step 3.1: Input the one-hot encoding X∈R of the HTTP text word. V ;

[0020] Step 3.2: Combine the one-hot encoding X of each word with the input weight matrix W∈R. V×N Multiply the input weight matrix W for all input words, then sum the resulting vectors and average them to obtain the hidden layer vector H∈R. N ;

[0021] Step 3.3: Multiply the hidden layer vector by the output weight matrix W'∈R V×NThe output vector is obtained and transformed into a probability distribution through the softmax activation function. The index position of the maximum probability is the predicted center word. During the training phase, the cross-entropy loss function is used to train the model and iteratively update the Word2Vec model.

[0022] Step 3.4: Multiply each input word by the shared input weight matrix W to obtain the word embedding vector. Use matrix W as a word embedding table T of the hybrid word embedding module. a ;

[0023] Step 3.5: Initialize the word embedding table T based on the embedding layer using Xavier_uniform uniform distribution. b Word embedding table T a It was trained before the gated Transformer model was trained; and T b The gated Transformer model is iteratively updated during training; the final hybrid word embedding table T f Table T is embedded by two words a and T b The result of average pooling is generated as shown in formula (1);

[0024] T f =(T a +T b ) / twenty one).

[0025] Furthermore, the specific method of step 4 is as follows:

[0026] Step 4.1: Before inputting the HTTP data text into the Transformer Encoder model, it needs to be processed by the hybrid word embedding module described in Step 3 to convert the text words into distributed numerical vector representations X. embedding ;

[0027] Step 4.2: Since the word order information of the words was not considered when inputting a series of word vectors, positional encoding information is periodically added to the text words using sine and cosine functions; the positional encoding information generated for each word position is then fused into the original text words, resulting in a word vector X after fusing positional encoding information. embedding-pe The generation methods are shown in formulas (2) and (3);

[0028] X embedding-pe =X embedding +X pos (2)

[0029]

[0030] Where pos represents the word order position of a word in the text, and its value ranges from 0 to the maximum length of the sequence; in order to incorporate positional encoding information, a positional encoding vector X is generated for a given word. pos The dimension of the word vector must be the same as the dimension of the word embedding. The word vector X is the result of the word being processed by the hybrid word embedding module. embedding The dimension and position encoding vector X pos The dimension of each is d emb Where 2i+1 and 2i represent word vectors X respectively. embedding and position encoding vector X pos In the odd and even positions, the range of values ​​for i is... d emb Representative word vector X embedding The dimension;

[0031] Step 4.3: The multi-head self-attention module extracts global sequence features from the text from multiple dimensions. The output dimension is the same as the input dimension, and each word in the text has been integrated with global features; for word vector X... embedding-pe Through three different linear mapping matrices W Q W K W V Three key pieces of information are generated, including the information to be queried (Q), the key of the word (K), and the value of the word (V), as shown in formula (4);

[0032]

[0033] When performing global attention feature extraction, attention scores are calculated using the question information Q to be extracted and the K corresponding to each word in the sentence. The calculation of attention scores is essentially the calculation of the correlation coefficient between words, and then the attention scores between words are used as weights to sum up V. This process is the principle of the self-attention mechanism. The scaling dot product used in the attention score calculation method is shown in formula (5).

[0034]

[0035] Among them, the denominator This is to prevent the dot product value from being too large, which would lead to an overly extreme value after passing through the softmax function. The subscript k represents the dimension of the Q, K, and V matrices.

[0036] Multi-head self-attention mechanism is for word vector X embedding-peThe self-attention mechanism is computed in different subspaces across multiple dimensions. When the self-attention mechanism needs to be computed in h different subspaces, the linear mapping matrix will be split into h blocks, and the h blocks of linear mapping matrices will correspond to the computation of the self-attention mechanism in each of the h different subspaces. Where s represents the attention of a certain subspace, s∈[1,h];

[0037] The output of the multi-head self-attention module is to extract HTTP text words from h different dimensional subspaces using a global attention mechanism, and then output the self-attention of h different heads as heads. s Feature concatenation is performed, and the multi-head self-attention mechanism outputs X. multihead The calculation method is shown in formula (6), where s represents the attention mechanism of a certain subspace;

[0038]

[0039] Step 4.4: Residual connection and layer normalization module;

[0040] The residual connection embeds the position-encoded word embedding vector X before the input multi-head self-attention module. embedding-pe The output of the multi-head self-attention module is added to the output of the multi-head self-attention module; the output data of the multi-head self-attention module is normalized using the LayerNorm method; the residual connection and layer normalization module are applied to the output X of the multi-head attention mechanism. multihead The calculation formula is shown in (7), and the output X of the multi-head self-attention module is... multihead The result X after residual connection and layer normalization processing multihead-rn That is, the final output of the residual connection and layer normalization modules;

[0041] X multihead-rn =LayerNorm(X embedding-pe +X multihead (7)

[0042] Step 4.5: Process the output X of the residual connection and layer normalization module through a fully connected neural network. multihead-rn Further processing extracts richer semantic information; ultimately, the output of the Transformer Encoder model is X. encoded The neural network calculation formula is shown in equation (8);

[0043]

[0044] Here, ReLU is a non-linear activation function.

[0045] Furthermore, the specific method of step 5 is as follows:

[0046] Step 5.1: Extract the output X from the Transformer Encoder model after global sequence feature extraction. encoded The input is fed into a gated convolutional module for information filtering and selection. The gated convolutional module includes c one-dimensional convolutional kernels of different scales. j (j∈[1,c]), the calculation formula for a single convolution kernel is shown in equation (9);

[0047] g j =Relu(Conv(Kernel) j∈c ,X encoded )+b j (9)

[0048] Among them, g j The output of a single convolutional block; ReLU is the non-linear activation function; Conv represents the convolution operation process; b j Kernel j The corresponding bias;

[0049] Step 5.2: Concatenate the outputs of convolutions at different scales, and map the values ​​of the multi-scale gated convolutions to the range of 0 to 1 using the Sigmoid activation function. This is the gate value Gatesv. The gate value Gatesv is calculated as shown in formula (10). The value of Gatesv is between 0 and 1. A gate value close to 0 means that the information is almost unimportant and will be filtered out. A gate value close to 1 means that the data is key information and will be retained.

[0050]

[0051] The output X of the Transformer Encoder model encoded Element-wise multiplication with the gate value Gatesv completes the encoding of information X. encoded The filtering process yields the output X of the gated convolution module. gated The information filtering method is shown in formula (11);

[0052] X gated =X encoded ⊙Gatesv,Gatesv∈[0,1] (11)

[0053] The ⊙ symbol represents element-level multiplication;

[0054] Furthermore, the specific method for step 6 is as follows:

[0055] The output of the gated convolution module X gatedThe input is fed into a classifier consisting of two fully connected layers. The output of the classifier is then converted into a probability distribution using a softmax function. Each dimension of the probability distribution corresponds to an attack category. The attack category corresponding to the index of the maximum probability value in the probability distribution is the final attack detection classification result X. pred The attack detection and classification process is shown in formula (12);

[0056] X pred =argmax(Softmax(Classifier(X gated ))) (12).

[0057] The beneficial effects of adopting the above technical solution are as follows: The Web attack detection method based on gated Transformer provided by this invention proposes a network model that can effectively extract multi-dimensional global and local features, improves the word embedding method, and the hybrid word vector table can contain more accurate and richer semantic information; it can automatically extract effective data information features from text sequences without the need for manual information filtering and word table replacement; it further improves the accuracy of multi-class attack detection and reduces the false positive rate, and can fully protect the security of the Web server system. Attached Figure Description

[0058] Figure 1 This is an overall architecture diagram of the network model provided in an embodiment of the present invention;

[0059] Figure 2 This is a structural diagram of the hybrid word embedding table model provided in an embodiment of the present invention;

[0060] Figure 3 This is a structural diagram of a gated convolutional network model provided in an embodiment of the present invention;

[0061] Figure 4 This is a simulation architecture diagram of a network attack dataset provided in an embodiment of the present invention;

[0062] Figure 5 This is an example diagram of a network attack payload provided in an embodiment of the present invention. Detailed Implementation

[0063] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0064] like Figure 1 The diagram shown is the overall network structure proposed by the method in this embodiment. The method of this embodiment is described below.

[0065] Step 1: Collect traffic using the sniff module of the Python scapy library to obtain a pcap traffic file and extract application layer data from it. The specific method is as follows:

[0066] Step 1.1: Start the sniff network interface sniffing module of the scapy library, set the network interface ifa to be monitored and the write_cap function to collect some network traffic and save it as a pcap file.

[0067] Step 1.2: Use the rd_pcap module of the Python scapy library to read and analyze the collected pcap files, extracting application-layer data. The application-layer data text to be analyzed includes the URL, parameter list, user-agent, cookie, and referer fields. Since injection attacks can also occur in the user-agent, cookie, and referer fields, the request parameter information in these fields cannot be ignored.

[0068] Step 2: Preprocess the message text by segmenting the URL, parameter list, user-agent, cookie, and referer fields using predefined special characters.

[0069] The message text is URL-decoded, and special characters such as " / &=.:?+ / / <>%()_" are used to segment the URL, parameter list, user-agent, cookie, and referer fields. During segmentation, it's necessary to consider the upper limit on the number of words after segmentation, as the Transformer Encoder model can only process a limited number of words in the input sequence. This is addressed by setting a maximum sentence length; words exceeding the maximum length are removed, and if the sentence length is less than the maximum, padding is used. In this implementation, the maximum length is set to 60, and padding is 0.

[0070] Step 3: The hybrid word embedding module enhances the robustness of vector representations by fusing word embedding tables generated in two different ways. The two word embedding tables in the hybrid word embedding module are a Word2Vec word embedding table based on the Continuous Bag-of-Words (Cbow) model and a word embedding table based on the embedding layer. The word embedding table based on the embedding layer is initialized using a Xavier_uniform distribution and its distributed representation of word vectors is continuously updated during training, while the word2Vec word embedding table needs to be generated before the model enters the training phase. The word embedding table based on the embedding layer and the word2Vec word embedding table map word vectors to different discrete spaces to achieve distributed vector representation of words. Compared with the traditional one-hot encoding method, this word embedding method has lower dimensionality and contains more semantic information. Using a hybrid word embedding table can more effectively learn the semantic relationships between HTTP text keywords, containing richer semantic information, such as... Figure 2 As shown. The specific method is as follows:

[0071] Step 3.1: Input the one-hot encoding X∈R of the HTTP text word. V ;

[0072] Step 3.2: Combine the one-hot encoding X of each word with the input weight matrix W∈R. V×N Multiply the input weight matrix W for all input words, then sum the resulting vectors and average them to obtain the hidden layer vector H∈R. N ;

[0073] Step 3.3: Multiply the hidden layer vector by the output weight matrix W'∈R V×N The output vector is obtained and transformed into a probability distribution using a softmax activation function. The index of the maximum probability value is the predicted center word. During the training phase, the cross-entropy loss function is used to train the model and iteratively update the Word2Vec model.

[0074] Step 3.4: Multiply each input word by the shared input weight matrix W to obtain the word embedding vector. Use matrix W as a word embedding table T of the hybrid word embedding module. a In this implementation scheme, the vocabulary Ta contains 2000 words, the word vector length is 300, min_count = 10, and window = 3.

[0075] Step 3.5: Initialize the word embedding table T based on the embedding layer using Xavier_uniform uniform distribution. b And T bIt will be iteratively updated simultaneously with the training process of the gated Transformer model, but the word embedding table T a It was trained before the gated Transformer model was trained. The final mixed word embedding table T f Table T is embedded by two words a and T b The average pooling result is generated as shown in formula (1). In this implementation, the word embedding table based on the Embedding layer also maps words to 300-dimensional vector representations, and the resulting word vectors have the same dimension as Ta.

[0076] T f =(T a +T b ) / twenty one)

[0077] Step 4: The text information of the HTTP message is processed by the hybrid word embedding module and converted into a series of word vectors. These word vectors are then input into the Transformer Encoder model for global attention feature extraction. The Transformer Encoder model consists of three parts: a positional encoding module, a multi-head self-attention module, and a residual layer normalization module. The Transformer Encoder model first adds positional encoding information to the vectors processed by the hybrid word embedding module, then extracts multi-dimensional sequence features through the multi-head self-attention mechanism module, and finally inputs it into the feedforward neural network module. In this implementation scheme, the Transformer Encoder model has two layers. Specific steps include:

[0078] Step 4.1: Before inputting the HTTP data text into the Transformer Encoder model, it needs to be processed by the hybrid word embedding module described in Step 3 to convert the text words into distributed numerical vector representations X. embedding In this implementation, X embedding A 300-dimensional vector;

[0079] Step 4.2: Since the word order information of the words was not considered when inputting a series of word vectors, positional encoding information is periodically added to the text words using sine and cosine functions; the positional encoding information generated for each word position is then fused into the original text words, resulting in a word vector X after fusing positional encoding information. embedding-pe The generation method is shown in formulas (2) and (3). Here, pos represents the word order position of a word in the text, and its value ranges from 0 to the maximum sequence length. To incorporate positional encoding information, a positional encoding vector X is generated for a given word. posThe dimension of the word vector must be the same as the dimension of the word embedding. The word vector X is the result of the word being processed by the hybrid word embedding module. embedding Dimension and position encoding X pos The dimension of each is d emb Where 2i+1 and 2i represent word vectors X respectively. embedding and position encoding vector X pos Given the odd and even positions in the array, the range of values ​​for i is [value missing]. Where d emb Representative word vector X embedding The dimension;

[0080] X embedding-pe =X embedding +X pos (2)

[0081]

[0082] Step 4.3: The multi-head self-attention mechanism module extracts global sequence features from multiple dimensions. The output has the same dimensions as the input data, but each word in the text has been integrated with global features. The calculation method of the attention mechanism borrows from the search query idea, and it applies global features to the word vector X. embedding-pe Through three different linear mapping matrices W Q W K W V Three key pieces of information are generated: the information to be queried (Q), the key of the word (K), and the value of the word (V), as shown in formula (4).

[0083]

[0084] When performing global attention feature extraction, it is only necessary to calculate the attention score by using the question information Q to be extracted and the K corresponding to each word in the sentence. The calculation of the attention score is essentially to calculate the correlation coefficient between words, and then use the attention scores between words as weights to perform a weighted summation on V. This process is the principle of the self-attention mechanism. The scaling dot product used in the attention score calculation method is shown in formula (5), where the denominator is... This is to prevent the dot product value from being too large, which would lead to an excessively extreme value after passing through the softmax function. The subscript k represents the dimension of the Q, K, and V matrices.

[0085]

[0086] Multi-head self-attention mechanism is for word vector X embedding-peThe self-attention mechanism is computed in different subspaces across multiple dimensions. When the self-attention mechanism needs to be computed in h different dimensional subspaces, the linear mapping matrix will be split into h blocks, each of which corresponds to the computation of the self-attention mechanism in one of the h different subspaces. Where s represents the attention of a certain subspace, s∈[1,h];

[0087] The output of the multi-head self-attention module is to extract HTTP text words from h different dimensional subspaces using a global attention mechanism, and then output the self-attention of h different heads as heads. s Feature concatenation is performed, and the multi-head self-attention mechanism outputs X. multihead The calculation method is shown in formula (6), where s represents the attention mechanism of a certain subspace. In this embodiment, the number of multi-head attention heads h = 6.

[0088]

[0089] Step 4.4: Residual Connectivity and LayerNorm Module;

[0090] The residual connection embeds the position-encoded word embedding vector X before the input multi-head self-attention module. emultihead-pe The output X of the multi-head self-attention module multihead In addition, residual connections can directly propagate gradients during backpropagation, thus avoiding gradient instability issues caused by excessive model depth. To accelerate model convergence and prevent gradient vanishing and exploding, layer normalization is used to normalize the data. The residual connections and layer normalization modules are then applied to the multi-head attention mechanism output X. multihead The calculation formula is shown in (7), and the output X of the multi-head self-attention module is... multihead The result X after residual connection and layer normalization processing multihead-rn That is, the final output of the residual connection and layer normalization modules.

[0091] X multihead-rn =LayerNorm(X embedding-pe +X multihead (7)

[0092] Step 4.5: Finally, process the output X of the multi-head self-attention module through a fully connected neural network (Linear). multihead Further processing extracts richer semantic information. Finally, the output of the Transformer Encoder is X. encoded The neural network calculation formula is shown in Equation (8), where Relu is a nonlinear activation function.

[0093]

[0094] Step 5: The HTTP text processed by the hybrid word embedding module undergoes global attention feature extraction via the Transformer Encoder model. The output of the Transformer Encoder model is then fed back into a gated convolutional model. This model extracts local features from the data within its receptive field. Non-critical information is dynamically filtered and selected using the gated convolutional model, such as... Figure 3 As shown, this model effectively solves the problem of traditional convolution treating all dimensions of the input data as valid data, automatically filtering data and further improving attack detection accuracy. Specific steps include:

[0095] Step 5.1: The output of the Transformer Encoder model after global sequence feature extraction is X. encoded X again encoded The input is fed into a gated convolutional module for information filtering and selection. The gated convolutional module includes c one-dimensional convolutional kernels of different scales. j (j∈[1,c]), the calculation formula for a single convolution kernel is shown in equation (9), where g j The output of a single convolutional block; ReLU is the non-linear activation function; Conv represents the convolution operation process; b j Kernel j The corresponding bias amount; in this embodiment, three convolution kernels with different sizes of 10, 15 and 25 are used.

[0096] g j =Relu(Conv(Kernel) j∈c ,X encoded )+b j (9)

[0097] Step 5.2: Convert the output g of convolutions at different scales i Feature concatenation is performed, and the values ​​of the multi-scale gated convolution are mapped to the range of 0 to 1 using the Sigmoid activation function. This is the gate value Gatesv. The calculation method for the gate value Gatesv is shown in formula (10). The value of Gatesv is between 0 and 1. A gate value close to 0 indicates that the information is almost unimportant and will be filtered out; a gate value close to 1 indicates that the data is key information and will be retained. The output X of the Transformer Encoder model is then processed. encoded Element-wise multiplication with the gate value Gatesv completes the encoding of information X. encoded The filtering process yields the output X of the gated convolution module.gated The information filtering method is shown in formula (11), where the ⊙ symbol represents element level multiplication.

[0098]

[0099] X gated =X encoded ⊙Gatesv,Gatesv∈[0,1] (11)

[0100] Step 6: Convert the output X of the gated convolution module gated The data is input into a classifier consisting of two fully connected layers, Linear1 and Linear2. These two layers initially increase the data dimensionality and then decrease it. Increasing the dimensionality represents the combination of various features, while decreasing it signifies the fusion of information from these combined features. Finally, the output of the fully connected network layers is converted into a probability distribution using a softmax function. Each dimension of the probability distribution corresponds to an attack category, and the attack category corresponding to the index of the maximum probability value is the final attack detection classification result X. pred The attack detection classification prediction method is shown in formula (12). In this implementation, the attack classification task is 10 categories.

[0101] X pred =argmax(Softmax(Lineart1(Lineart2(X gated (12)

[0102] The proposed gated Transformer-based network model in this embodiment combines the Transformer and gated convolution modules. The Transformer extracts global semantic information across different spatial dimensions through a multi-head self-attention mechanism, while the gated convolution extracts information in the local space through a one-dimensional convolution kernel. Furthermore, a gating mechanism is used to filter and select textual information. This model has the following main advantages:

[0103] (1) The Transformer model was used and improved, which has the following advantages: The word embedding layer of Transformer is initialized and average pooling is performed in two ways, namely word2vec based on cbow and xavier_uniform initialization, which can train word vectors more fully and improve robustness; the multi-head self-attention mechanism can effectively extract multi-dimensional global sequence features. Compared with other sequence models such as RNN and LSTM, it has no temporal dependence, can use GPU parallel computing to shorten training time, and will not generate too much computational complexity.

[0104] (2) The gated convolution module further extracts n-gram local features and performs effective information masking and filtering. It has the following advantages: The convolutional gate unit uses convolution operations to process the data output by Transformer. Transformer's multi-head self-attention mechanism extracts global information features from multiple spatial dimensions, but the extraction of local features may be somewhat lacking. Since the parameters in the URL are n-gram local features of type param=value, the gated convolution with shared parameters can extract local feature information more fully; Multiple one-dimensional convolutional kernels of different scales can more effectively deal with words of different lengths. Finally, the output vectors of convolutional kernels of different scales are concatenated to avoid the inability of a single-scale convolutional kernel to fully match words of different lengths, and to avoid information loss due to insufficient extraction of local features; For relatively long HTTP messages, which contain many symbols, numbers, etc. that do not contain information, there is no need to use a replacement rule dictionary to reduce the vocabulary space. It automatically filters and extracts key and effective information in complex information while ignoring irrelevant information, further improving the accuracy of attack detection.

[0105] This embodiment conducted test experiments on the publicly available dataset CSIC2010 and network traffic collected from simulated network attacks. The HTTP CSIC 2010 dataset was provided as an appendix in a paper by the Spanish Supreme Scientific Council (CSIC). The collected network traffic consisted of records of normal access and network attacks on an e-commerce website, including 36,000 normal requests and over 25,000 attack requests. The abnormal request samples included attack samples of various categories such as SQL injection, file traversal, CRLF injection, XSS, and SSI. To improve the model's generalization ability and verify its performance in real-world network environments, and to more fully learn the characteristics of various attack categories, network attack simulation experiments were conducted using Kali Linux. The traffic generated by the attacks was collected, with the main simulated access target being a web website, including both normal access requests and attack requests. The attack simulation experiments are as follows: Figure 4 As shown, the victim continuously ran Wireshark to collect and store traffic. The attacker targeted a website hosted on the target host, primarily using tools such as sqlmap, nmap, and Metasploit Framework. The attack included 50,000 legitimate requests and 100,000 malicious requests. The data was categorized into 10 tags: normal, abnormal, SQL injection, buffer overflow, format string, SSI, XPath, XSS, CRLFi, and LDAP injection. Detailed attack payloads are shown below. Figure 5 As shown.

[0106] This embodiment adopts three performance evaluation metrics commonly used in attack detection systems, including accuracy, recall, and F1 score, and the calculation formulas are shown in equations (11)-(13).

[0107]

[0108]

[0109]

[0110] Comparative experiments were conducted with multiple baseline models, including CNN, LSTM, and BiLSTM, on the network traffic collected from the CSIC2010 dataset and simulated network attacks. The accuracy, recall, and F1 score of the various models were evaluated in the experiments. The results of the attack detection ten-class experiment are shown in Tables 1 and 2. The experimental data used the traffic generated from the CSIC2010 dataset and the simulated attack, respectively.

[0111] Table 1. Experimental results of attack detection based on the CSIC2010 dataset (10 categories)

[0112]

[0113]

[0114] Table 2. Experimental results of attack detection based on a network attack simulation dataset (10 categories)

[0115] Model Accuracy F1 Rec DT 87.87％ 89.35％ 94.54％ Linear SVM 87.23％ 88.51％ 88.15％ BiLSTM+CNN 94.54％ 94.12％ 94.98％ BiLSTM 93.15％ 91.34％ 93.46％ CNN 92.87％ 93.61％ 94.31％ LSTM+CNN 93.43％ 93.51％ 93.63％ LSTM 91.71％ 92.8％ 92.96％ LSTM+GatedCNN 93.15％ 92.31％ 93.54％ Transformer 94.43％ 94.32％ 94.45％ Gated Transformer 96.64％ 96.51％ 97.54％

[0116] The method of this invention outperforms the comparative experimental model in all three of the above indicators, demonstrating that the invention has a significant improvement in attack detection effectiveness. At the same time, the comparative experiment shows that the improved gated Transformer network model can effectively extract global and local features and can automatically extract effective features from text sequences, eliminating the need for manual HTTP keyword table replacement and effectively protecting the security of the web server system.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A Web attack detection method based on gated Transformer, characterized in that: Includes the following steps: Step 1: Collect traffic using the sniff module of the Python scapy library, obtain a pcap traffic file, and extract application layer data from it; Step 2: Perform URL decoding on the message text of the application layer data, and segment the text information of the URL, parameter list, user-agent, cookie, and referer fields using predefined special characters; Step 3: The hybrid word embedding module enhances the robustness of the vector representation by fusing word embedding tables generated in two different ways; the two word embedding tables in the hybrid word embedding module are the Word2Vec word embedding table based on the continuous bag-of-words model Cbow and the word embedding table based on the Embedding layer. The word embedding table based on the Embedding layer is initialized through the Xavier_uniform distribution and will be continuously updated with the distributed representation of word vectors during training; the Word2Vec word embedding table based on the continuous bag-of-words model Cbow needs to be generated before the model enters the training phase. The word embedding table based on the Embedding layer and the Word2Vec word embedding table based on the continuous bag-of-words model Cbow respectively map word vectors to different discrete spaces, thereby performing distributed vector representation of words; Step 4: After the text information of the HTTP message is processed by the hybrid word embedding module in Step 3 and converted into a series of word vectors, that is, the distributed vector representation of the words is obtained, the series of word vectors are input into the TransformerEncoder model for global attention feature extraction. The Transformer Encoder model consists of three parts: a positional encoding module, a multi-head self-attention module, and a residual layer normalization module. The Transformer Encoder model first adds positional encoding information to the vector processed by the hybrid word embedding module, then extracts multi-dimensional sequence features through the multi-head self-attention mechanism module, and finally inputs it into the feedforward neural network module. Step 5: Input the output of the Transformer Encoder model into the gated convolutional model. The gated convolutional model extracts local features from the data within the local receptive field. The gated convolutional model dynamically filters and selects non-critical data. Step 6: Classify the output of the gated convolution model through the final classifier module; the softmax function converts it into a probability distribution, each dimension of the probability distribution corresponds to an attack category, and the attack category corresponding to the index of the maximum probability in the probability distribution is the final attack detection classification result.

2. The Web attack detection method based on gated Transformer according to claim 1, characterized in that: The specific method for step 1 is as follows: Step 1.1: Start the sniff network interface sniffing module of the Python scapy library to collect network traffic and save it as a pcap file; Step 1.2: Use the rd_pcap module of the Python scapy library to read and analyze the collected pcap files, extract application layer data from the pcap files and perform data text analysis, including text information of URL, parameter list, user-agent, cookie, and referer fields.

3. The Web attack detection method based on gated Transformer according to claim 1, characterized in that: In the word segmentation process in step 2, since the Transformer Encoder model can only process a limited number of words in the input sequence, there is an upper limit to the number of words after text segmentation. This is addressed by setting a maximum sentence length. Words exceeding the maximum sentence length will be removed. If the number of words in a sentence is less than the maximum length, padding will be used to fill the gaps.

4. The Web attack detection method based on gated Transformer according to claim 1, characterized in that: The specific method for step 3 is as follows: Step 3.1: Input one-hot encoding of HTTP text words ; Step 3.2: Encode each word using one-hot encoding respectively with the input weight matrix Multiplication is performed, where the input weight matrix W is shared across all input words. The resulting vectors are then summed and averaged to form the hidden layer vector. ; Step 3.3: Multiply the hidden layer vector by the output weight matrix The output vector is obtained and transformed into a probability distribution through the softmax activation function. The index position of the maximum probability is the predicted center word. During the training phase, the cross-entropy loss function is used to train the model and iteratively update the Word2Vec model. Step 3.4: Multiply each input word by the shared input weight matrix W to obtain the word embedding vector. Use matrix W as a word embedding table T of the hybrid word embedding module. a ; Step 3.5: Initialize the word embedding table T based on the embedding layer using Xavier_uniform uniform distribution. b Word embedding table T a It was trained before the gated Transformer model was trained; and T b The gated Transformer model is iteratively updated during training; the final hybrid word embedding table T f Table T is embedded by two words a and T b The result of average pooling is generated as shown in formula (1); (1)。 5. The Web attack detection method based on gated Transformer according to claim 1, characterized in that: The specific method for step 4 is as follows: Step 4.1: Before inputting the HTTP data text into the Transformer Encoder model, it needs to be processed by the hybrid word embedding module described in Step 3 to convert the text words into distributed numerical vector representations. ; Step 4.2: Since the word order information of the words themselves is not taken into account when inputting a series of word vectors, positional encoding information is periodically added to the text words using sine and cosine functions; The positional encoding information generated for each word at each position is fused into the original text words, resulting in a word vector after fusing the positional encoding information. The generation methods are shown in formulas (2) and (3); (2)； (3)； in, This represents the word order position of a word in the text, and its value ranges from 0 to an integer between the maximum length of the sequence; it is the positional encoding vector generated for a given word in order to incorporate positional encoding information. The dimension of the word vector must be the same as the dimension of the word embedding. The word vector is the result of the word embedding module. Dimension and position encoding vector All dimensions are ,in They represent word vectors respectively. and position encoding vector In the odd and even positions, the range of values ​​for i is... , Representative word vectors The dimension; Step 4.3: The multi-head self-attention module extracts global sequence features from the text from multiple dimensions. The output dimension is the same as the input data dimension, and each word in the text has been integrated with global features; for word vectors... Through three different linear mapping matrices Three key pieces of information are generated, including the information to be queried (Q), the key of the word (K), and the value of the word (V), as shown in formula (4); (4)； When performing global attention feature extraction, attention scores are calculated using the question information Q to be extracted and the K corresponding to each word in the sentence. The calculation of attention scores is essentially the calculation of the correlation coefficient between words, and then the attention scores between words are used as weights to sum up V. This process is the principle of the self-attention mechanism. The scaling dot product used in the attention score calculation method is shown in formula (5). (5)； Among them, the denominator This is to prevent the dot product from being too large, which would lead to... The value after the function is too extreme, the subscript represent ; Multi-head self-attention mechanism is for word vectors The self-attention mechanism is computed from different subspaces across multiple dimensions; when it is necessary to compute from... When performing self-attention mechanism computation in subspaces of different dimensions, the linear mapping matrix will be decomposed into... Block, after splitting The block linear mapping matrices correspond to respectively Computation of self-attention mechanisms in different subspaces , , , where s represents the attention of a certain subspace, ; The output of the multi-head self-attention module is to extract HTTP text words from h different dimensional subspaces using a global attention mechanism, and then output the self-attention from h different heads. Feature concatenation is performed, and the output is generated by a multi-head self-attention mechanism. The calculation method is shown in formula (6), where s represents the attention mechanism of a certain subspace; (6)； Step 4.4: Residual connection and layer normalization module; The residual connection encodes the word embedding vectors located before the input multi-head self-attention module. The output of the multi-head self-attention module is added to the output of the multi-head self-attention module; the output data of the multi-head self-attention module is normalized using the LayerNorm method; the residual connection and layer normalization module are then applied to the output of the multi-head attention mechanism. The calculation formula is shown in (7), and the output of the multi-head self-attention module is... The result after residual connection and layer normalization processing That is, the final output of the residual connection and layer normalization modules; (7)； Step 4.5: Process the output of the residual connection and layer normalization modules using a fully connected neural network. Further processing extracts richer semantic information; ultimately, the output of the Transformer Encoder model is... The neural network calculation formula is shown in equation (8); (8)； in, It is a non-linear activation function.

6. The Web attack detection method based on gated Transformer according to claim 5, characterized in that: The specific method for step 5 is as follows: Step 5.1: Extract the output of the Transformer Encoder model after global sequence feature extraction. The input is fed into a gated convolutional module for information filtering and selection. The gated convolutional module includes c one-dimensional convolutional kernels of different scales. The formula for calculating a single convolution kernel is shown in equation (9); (9)； in, The output of a single convolutional block; It is a non-linear activation function; This represents the convolution operation process; convolution kernel The corresponding bias; Step 5.2: Concatenate the features of the outputs from convolutions at different scales, and then... The activation function maps the values ​​of multi-scale gated convolutions to... The range is the threshold value. Gating value The calculation method is as follows: As shown, The range of values ​​is within Between; a gating value close to 0 indicates that the information is almost unimportant and will be filtered out; a gating value close to 1 indicates that the information is critical and will be retained. ； The output of the Transformer Encoder model and Perform element-level multiplication to complete the encoding of information. The filtering process yields the output of the gated convolution module. The information filtering method is shown in formula (11); (11)； in, The symbol represents element-level multiplication.

7. The Web attack detection method based on gated Transformer according to claim 6, characterized in that: The specific method for step 6 is as follows: The output of the gated convolution module The input is fed into a classifier consisting of two fully connected layers. The output of the classifier network is then converted into a probability distribution using a softmax function. Each dimension of the probability distribution corresponds to an attack category, and the attack category corresponding to the index of the maximum probability value in the probability distribution is the final attack detection classification result. The attack detection and classification process is shown in formula (12); (12)。

Images