A Transformer-based method for detecting unknown web attacks
By constructing a sequence autoencoder based on a Transformer-based unsupervised learning model, and utilizing a multi-layer Transformer encoder and a GRU decoder, the problem of low accuracy in the detection of unknown Web attacks is solved, achieving efficient identification of unknown attacks and modeling of normal behavior patterns.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2024-09-02
- Publication Date
- 2026-05-26
Smart Images
Figure CN119210792B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of cybersecurity, and in particular to a method for detecting unknown web attacks based on Transformer. Background Technology
[0002] With the development of network technology, Web applications have become one of the most important applications on the Internet. They can be used for website backends or to provide application programming interfaces (APIs) for various other services. Currently, various organizations, including large Internet companies and government agencies, provide Web application services. However, while providing large-scale services, attacks targeting Web applications are also emerging in large numbers, such as SQL injection and cross-site scripting (XSS) attacks. Attackers launch attacks by sending carefully crafted malicious requests to Web servers, which can lead to various serious consequences such as arbitrary file read / write and malicious command execution.
[0003] The current mainstream approach to defending against web attacks is to deploy a Web Application Firewall (WAF). Any web request is only submitted to the web server for processing after being deemed legitimate by the WAF's security mechanisms; otherwise, it is blocked by the WAF. Traditional WAFs are rule-based attack detection systems. They can only match known attack methods by manually adding rules and cannot detect unknown attacks. Furthermore, they rely on a high-quality rule base, which typically requires professional security operations personnel to write and maintain it regularly, such as adding and modifying rules based on threat intelligence. WAFs also have limitations on the number of rules that can be configured; too many rules significantly increase computational overhead, leading to low detection efficiency.
[0004] Besides rule-based detection, machine learning methods have also been widely used for web attack detection in recent years due to the development of machine learning technology. These methods can be divided into supervised and unsupervised approaches. Supervised methods require training data to include at least two categories of labeled data: normal and attack. Their effectiveness is highly dependent on the quality of the labeled data. Specifically, supervised methods limit prior knowledge about attacks in the training data, thus they still cannot identify unknown attacks outside the training data distribution. Unsupervised methods use the idea of anomaly detection, requiring only data from the normal category to model the normal behavior patterns of web applications. Samples deviating from normal request patterns are identified as attacks. However, existing unsupervised methods do not consider the structural information of HTTP requests, thus failing to effectively model the normal behavior patterns of web applications, resulting in typically low detection accuracy. Summary of the Invention
[0005] This application aims to at least partially address one of the technical problems in the related art.
[0006] Therefore, the purpose of this application is to propose a Transformer-based method for detecting unknown Web attacks. Through unsupervised learning, it can detect unknown Web attacks without relying on manual rules or labeled data. Furthermore, by proposing a new unsupervised learning model based on an improved Transformer, it can effectively capture the structural information of HTTP requests and model the normal behavior patterns of Web applications.
[0007] To achieve the above objectives, this application proposes a Transformer-based method for detecting unknown web attacks, including:
[0008] S1: Collect HTTP requests in real time and preprocess them into a tag sequence;
[0009] S2: Insert a special marker at the beginning of the marker sequence, convert each marker in the expanded marker sequence into an embedded representation, and encode the embedded representation using a multi-layer Transformer encoder to obtain an intermediate representation for each marker;
[0010] S3: Use the intermediate representation as the initial hidden state of the GRU decoder, and generate a reconstructed tag sequence by performing sequence decoding on the GRU decoder;
[0011] S4: By comparing the reconstruction error between the original tag sequence and the reconstructed tag sequence, determine whether a given HTTP request conforms to the pattern characteristics of a normal HTTP request.
[0012] Optionally, the real-time collection of HTTP requests and preprocessing them into a token sequence includes:
[0013] According to the HTTP protocol, extract the request method, URL path, URL request parameters and request body from each HTTP request, and combine the extracted information into a string according to a specific template;
[0014] The string is segmented using reserved characters, English punctuation characters, and whitespace characters based on the HTTP protocol to obtain the token sequence.
[0015] Optionally, inserting a special marker at the beginning of the marker sequence and converting each marker in the expanded marker sequence into an embedded representation includes:
[0016] For the labeled sequence X = { <encoder>x1, x2, ..., x n }, insert a special marker at the beginning of the marker sequence. <encoder>The expanded label sequence { is obtained <encoder>x1, x2, ..., x n };
[0017] For the expanded tag sequence { <encoder>x1, x2, ..., x n For each token in the array, convert it into an embedding representation consisting of word embeddings and positional embeddings, expressed as:
[0018]
[0019] Here, 'e' is a combination of word embedding and positional embedding.
[0020] Optionally, the step of encoding the embedded representation using a multi-layer Transformer encoder to obtain an intermediate representation for each tag includes:
[0021] The embedded representation is encoded using a multi-layer Transformer encoder to obtain a context representation vector for each tag, expressed as:
[0022]
[0023] Here, let L be the total number of layers in the multi-layer Transformer encoder, and H represent the context representation vector for each tag, H( L ) represents the calculation result of the Lth layer, where h ＜encoder> As an intermediate representation for each tag.
[0024] Optionally, the calculation result of each layer in the multi-layer Transformer encoder is represented as follows:
[0025] H (l) =TransformerLayer (l) (H (l-1) ),
[0026] Where l = 1, 2, ..., L, H (0) =E, where TransformerLayer represents the computation function of each Transformer layer, including a self-attention mechanism and a feedforward neural network. The computational expression for the self-attention mechanism is:
[0027]
[0028] Where Q, K, and V are derived from the input H (l-1) Query, key, and value matrices obtained through different linear transformations;
[0029] The output of the self-attention mechanism is further processed by a feedforward neural network, and the expression is:
[0030] FFN(H)=ReLU(HW1+b1)W2+b2,
[0031] Where W1, W2, b1, and b2 are trainable parameters.
[0032] Optionally, the step of using the intermediate representation as the initial hidden state of the GRU decoder and generating a reconstructed tag sequence by performing sequence decoding on the GRU decoder includes:
[0033] The intermediate representation is h <enncoder>< / enncoder> h0 serves as the initial hidden state for the GRU decoder;
[0034] At each time step t, the GRU decoder uses the tag y generated in the previous time step. t-1 The hidden state h of the previous time step t-1 Generate a new hidden state h t The expression is:
[0035] h t =GRUCell(y t-1 h t-1 )
[0036] The hidden state h at each time step t By mapping a linear layer to the output dimension, the probability distribution of the output tags is obtained, and a reconstructed tag sequence is generated by sampling from the probability distribution.
[0037] Optionally, steps S2 and S3 are performed on the trained Transformer-based unsupervised autoencoder model, wherein the training process of the Transformer-based unsupervised autoencoder model is as follows:
[0038] Collect log data from a web application over a period of time, and extract all plaintext HTTP requests from the log data during that period as raw training data.
[0039] According to the HTTP protocol, the request method, URL path, URL request parameters and request body are extracted from each HTTP request in the original training data, and the extracted information is combined into a string sample according to a specific template;
[0040] The string sample is segmented based on reserved characters, English punctuation characters, and whitespace characters in the HTTP protocol to obtain a tokenized sequence sample;
[0041] The sequence autoencoder model is trained using labeled sequence samples. The training objective is to make the reconstructed labeled sequence samples output by the model as consistent as possible with the original labeled sequence samples. The loss function is as follows:
[0042]
[0043] Where T j f represents the one-hot encoding corresponding to the j-th labeled sequence sample in the dataset. θ (T j Let R represent the probability distribution output by the model for the j-th labeled sequence sample, and let R be the negative log-likelihood function.
[0044] After calculating the loss function, the model parameters of the sequence autoencoder model are iteratively updated using the gradient descent method.
[0045] Optionally, determining whether a given HTTP request conforms to the pattern characteristics of a normal HTTP request by comparing the reconstruction error between the original marker sequence and the reconstructed marker sequence includes:
[0046] Calculate the BLEU score between the original tag sequence and the reconstructed tag sequence. If the BLEU score is lower than a preset threshold, the detected HTTP request is determined to be an abnormal attack request; otherwise, it is a normal request.
[0047] Optionally, the preset threshold can be adjusted based on the false alarm rate and false negative rate in the actual application scenario.
[0048] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:
[0049] By using unsupervised learning, we can detect unknown web attacks without relying on manual rules or labeled data. Furthermore, by proposing a new unsupervised learning model based on an improved Transformer, we can effectively capture the structural information of HTTP requests and model the normal behavior patterns of web applications.
[0050] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0051] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0052] Figure 1 This is a flowchart illustrating an unknown web attack detection method based on Transformer, according to an embodiment of this application;
[0053] Figure 2 This is an architecture diagram of an unsupervised autoencoder model based on Transformer, as shown in the embodiments of this application. Detailed Implementation
[0054] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0055] The core idea of this application is to model the semantics of normal HTTP requests by constructing a sequence autoencoder model. The sequence autoencoder includes two main modules: an encoder and a decoder. The encoder takes the preprocessed original token sequence of the original HTTP request as input and encodes it into an intermediate representation in the latent space. Then, the decoder attempts to reconstruct the original token sequence from the intermediate representation. Finally, the reconstruction token error between the original token sequence and the reconstructed sequence is compared to determine whether a given HTTP request conforms to the pattern characteristics of a normal HTTP request.
[0056] Because HTTP requests inherently possess structural information, such as the correlation between URL parameters, the dependencies between different fields need to be considered when encoding the original sequence to ensure effective modeling. The Transformer architecture helps address this issue. Unlike traditional sequence models such as recurrent neural networks, the Transformer treats the entire sequence as a whole, capturing the dependencies between any two labels through a self-attention mechanism. Furthermore, compared to traditional sequence models like recurrent neural networks, it can process data in parallel, further increasing detection efficiency.
[0057] However, the Transformer's built-in encoding and decoding mechanism cannot be used to train sequence autoencoder models because its decoders use a cross-attention structure, taking the encoder's output at each time step as input. This process does not compress the feature dimension, thus failing to effectively learn the latent space feature representation. This results in a decrease in the model's ability to detect attacks. Therefore, this application proposes a novel Transformer-based unsupervised autoencoder model. This model first inserts a special "..." at the beginning of the original labeled sequence. <encoder>"The new tag sequence, after being encoded by the Transformer,..." <encoder>"The representation vector corresponding to the marker is extracted for decoding; in the decoding stage, this application abandons the inherent decoder of the Transformer and instead chooses to use a gated recurrent unit (GRU) for decoding. This application uses..." <encoder>The representation vector corresponding to the tag is used as the initial state of GRU, thereby generating the reconstructed tag sequence.
[0058] The following describes in detail, with reference to the accompanying drawings, an embodiment of the present application of a method for detecting unknown web attacks based on Transformer.
[0059] Figure 1 This is a flowchart illustrating a Transformer-based method for detecting unknown web attacks according to embodiments of this application, as shown below. Figure 1 As shown, the method includes the following steps:
[0060] Step S1: Collect HTTP requests in real time and preprocess them into a tag sequence.
[0061] In this embodiment of the application, the following preprocessing steps are performed for each collected HTTP request:
[0062] According to the HTTP protocol, the request method, URL path, URL request parameters and request body are extracted from each HTTP request, and the extracted information is combined into a string according to a specific template; the string is segmented based on reserved characters, English punctuation characters and whitespace characters of the HTTP protocol to obtain a token sequence.
[0063] In one possible embodiment, a specific template can be represented as:
[0064] "Method:{request method}Path:{URL path}Para:{URL request parameters}body:{request body}"
[0065] It is understandable that whitespace characters include newline characters and space characters, etc.
[0066] Step S2: Insert a special token at the beginning of the token sequence, convert each token in the expanded token sequence into an embedded representation, and encode the embedded representation using a multi-layer Transformer encoder to obtain an intermediate representation for each token.
[0067] It is understandable that the Transformer's proprietary encoding and decoding mechanism cannot be used to train sequence autoencoder models because its decoders use a cross-attention structure that takes the encoder's output at each time step as input. This process does not compress the feature dimension, thus failing to effectively learn the latent space feature representation. This results in a decrease in the model's ability to detect attacks. Therefore, in this embodiment, for the labeled sequence X = { <encoder>x1, x2, ..., x n }, insert a special marker at the beginning of the marker sequence. <encoder>The expanded label sequence { is obtained <encoder>x1, x2, ..., x n }; and, for the expanded tag sequence { <encoder>x1, x2, ..., x n Each tag in} (including <encoder>This is converted into an embedding representation consisting of word embeddings and positional embeddings, expressed as:
[0068]
[0069] Here, 'e' is a combination of word embedding and positional embedding.
[0070] Furthermore, the embedded representation is encoded using a multi-layer Transformer encoder.
[0071] In some embodiments of this application, if the total number of layers in the multi-layer Transformer encoder is L, then the calculation result for each layer is expressed as follows:
[0072] H (l) =TransformerLayer (l) (H (l-1) ),
[0073] Where l = 1, 2, ..., L, H (0) =E, TransformerLayer represents the operation function of each Transformer layer, including self-attention mechanism and feedforward neural network.
[0074] The computational expression for the self-attention mechanism is:
[0075]
[0076] Where Q, K, and V are derived from the input H ( l-1 The query, key, and value matrices obtained through different linear transformations.
[0077] Furthermore, the output of the self-attention mechanism is further processed by a feedforward neural network, and the expression is:
[0078] FFN(H)=ReLU(HW1+b1)W2+b2,
[0079] Where W1, W2, b1, and b2 are trainable parameters.
[0080] Finally, after encoding by multiple Transformer encoders, the output is a context representation vector for each tag (including...). <encoder>),Right now:
[0081]
[0082] Where H represents the context representation vector for each tag, H( L ) represents the calculation result of the Lth layer.
[0083] In this embodiment of the application, h ＜enncoder> As an intermediate representation for each tag.
[0084] Step S3: Use the intermediate representation as the initial hidden state of the GRU decoder, and generate the reconstructed tag sequence by performing sequence decoding on the GRU decoder.
[0085] It should be noted that this application abandons the inherent decoder of Transformer in the decoding stage and instead chooses to use a gated loop unit (GRU) for decoding.
[0086] Specifically, first, the intermediate representation h... <encoder>< / encoder> The initial hidden state h0 is used as the GRU decoder; then at each time step t, the GRU decoder uses the tag y generated in the previous time step. t-1 The hidden state h of the previous time step t-1 Generate a new hidden state h t The expression is:
[0087] h t =GRUCell(y t-1 h t-1 )
[0088] Finally, the hidden state h at each time step t By mapping a linear layer to the output dimension, the probability distribution of the output labels is obtained, and a reconstructed label sequence is generated by sampling from the probability distribution.
[0089] It is understood that steps S2 and S3 above are essentially the online deployment phase of the Transformer-based unsupervised autoencoder model proposed in this application, responsible for predicting real-time HTTP traffic to identify attacks. The architecture diagram of the Transformer-based unsupervised autoencoder model is shown below. Figure 2 As shown.
[0090] In this embodiment of the application, the training process of the Transformer-based unsupervised autoencoder model is as follows:
[0091] Log data from a web application over a specific period is collected. All plaintext HTTP requests within that period are extracted as raw training data. For each HTTP request in the raw training data, the request method, URL path, URL request parameters, and request body are extracted according to the HTTP protocol. This extracted information is then combined into string samples using a specific template. The string samples are segmented based on reserved characters, punctuation characters, and whitespace characters in the HTTP protocol to obtain labeled sequence samples. The labeled sequence samples are then used to train the sequence autoencoder model. The training objective is to ensure that the reconstructed labeled sequence samples output by the model are as consistent as possible with the original labeled sequence samples. The loss function is as follows:
[0092]
[0093] Where T j f represents the one-hot encoding corresponding to the j-th labeled sequence sample in the dataset. θ (T j Let R represent the probability distribution output by the model for the j-th labeled sequence sample, and let R be the negative log-likelihood function. After calculating the loss function, the model parameters of the sequence autoencoder model are iteratively updated according to the gradient descent method.
[0094] In one possible embodiment, a specific template during the training process can be represented as:
[0095] "Method: {request method} Path: {URL path} Para: {URL request parameters} body: {request body}"
[0096] It is understandable that whitespace characters include newline characters and space characters, etc.
[0097] Step S4: By comparing the reconstruction error between the original tag sequence and the reconstructed tag sequence, determine whether the given HTTP request conforms to the pattern characteristics of a normal HTTP request.
[0098] Finally, after obtaining the reconstructed marker sequence through the above steps, the difference between the two sequences is measured by calculating the BLEU score between the original marker sequence and the reconstructed marker sequence.
[0099] Furthermore, when the BLEU score is lower than a preset threshold, the detected HTTP request is determined to be an abnormal attack request; otherwise, it is a normal request.
[0100] The preset threshold is adjusted according to the false alarm rate and false negative rate in the actual application scenario. This application does not specify the specific value of the preset threshold.
[0101] In summary, this invention addresses real-world web applications and proposes an unsupervised detection method for unknown web attacks, enabling attack detection without acquiring any prior knowledge about the attack. Furthermore, model training and deployment can be completed solely by collecting log data from the web application, eliminating the need for manual annotation. Moreover, the Transformer-based unsupervised autoencoder model proposed in this invention effectively considers the structural information of HTTP requests, thereby effectively learning the latent space intermediate representation of HTTP requests and achieving modeling of normal HTTP request patterns.
[0102] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.
[0103] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.< / encoder> < / encoder> < / encoder> < / encoder> < / encoder> < / encoder> < / encoder> < / encoder> < / encoder> < / encoder> < / encoder> < / encoder> < / encoder>
Claims
1. A method for detecting unknown web attacks based on Transformer, characterized in that, include: S1: Collect HTTP requests in real time and preprocess them into a tag sequence; S2: Insert a special marker at the beginning of the marker sequence, convert each marker in the expanded marker sequence into an embedded representation, and encode the embedded representation using a multi-layer Transformer encoder to obtain an intermediate representation for each marker; S3: Using the intermediate representation as the initial hidden state of the GRU decoder, and generating a reconstructed tag sequence by performing sequence decoding on the GRU decoder, including: using the intermediate representation... As the initial hidden state of the GRU decoder ; at each time step The GRU decoder uses the tags generated in the previous time step. The hidden state of the previous time step Generate new hidden states The expression is: The hidden state at each time step By mapping a linear layer to the output dimension, the probability distribution of the output tags is obtained, and a reconstructed tag sequence is generated by sampling from the probability distribution. S4: By comparing the reconstruction error between the original tag sequence and the reconstructed tag sequence, determine whether a given HTTP request conforms to the pattern characteristics of a normal HTTP request.
2. The method according to claim 1, characterized in that, The real-time collection of HTTP requests and their preprocessing into a tag sequence includes: According to the HTTP protocol, extract the request method, URL path, URL request parameters and request body from each HTTP request, and combine the extracted information into a string according to a specific template; The string is segmented using reserved characters, English punctuation characters, and whitespace characters based on the HTTP protocol to obtain the token sequence.
3. The method according to claim 1, characterized in that, The step of inserting a special marker at the beginning of the marker sequence and converting each marker in the expanded marker sequence into an embedded representation includes: For the labeled sequence Insert a special marker at the beginning of the marker sequence. The expanded marker sequence is obtained. ; For the expanded tag sequence For each token, convert it into an embedding representation composed of word embeddings and positional embeddings, expressed as: , in, It is a combination of word embedding and positional embedding.
4. The method according to claim 3, characterized in that, The process of encoding the embedded representation using a multi-layer Transformer encoder to obtain an intermediate representation for each tag includes: The embedded representation is encoded using a multi-layer Transformer encoder to obtain a context representation vector for each tag, expressed as: Wherein, the total number of layers in the multi-layer Transformer encoder is . , This represents the context representation vector for each tag. Indicates the first The calculation results of the layer will As an intermediate representation for each tag.
5. The method according to claim 4, characterized in that, The calculation result of each layer in the multi-layer Transformer encoder is represented as follows: ， in, , , This represents the computation function for each Transformer layer, including a self-attention mechanism and a feedforward neural network. The computational expression for the self-attention mechanism is as follows: ， in, It is input Query, key, and value matrices obtained through different linear transformations; The output of the self-attention mechanism is further processed by a feedforward neural network, and the expression is: ， in, , , , These are trainable parameters.
6. The method according to claim 5, characterized in that, Steps S2 and S3 are executed after the Transformer-based unsupervised autoencoder model has been trained. The training process of the Transformer-based unsupervised autoencoder model is as follows: Collect log data from a web application over a period of time, and extract all plaintext HTTP requests from the log data during that period as raw training data. According to the HTTP protocol, the request method, URL path, URL request parameters and request body are extracted from each HTTP request in the original training data, and the extracted information is combined into a string sample according to a specific template; The string sample is segmented based on reserved characters, English punctuation characters, and whitespace characters in the HTTP protocol to obtain a tokenized sequence sample; The sequence autoencoder model is trained using labeled sequence samples. The training objective is to make the reconstructed labeled sequence samples output by the model as consistent as possible with the original labeled sequence samples. The loss function is as follows: in Represents the first in the dataset The one-hot encoding corresponding to each labeled sequence sample Indicates the model for the first The probability distribution output by a labeled sequence sample, function It is a negative log-likelihood function; After calculating the loss function, the model parameters of the sequence autoencoder model are iteratively updated using the gradient descent method.
7. The method according to claim 1, characterized in that, The step of determining whether a given HTTP request conforms to the pattern characteristics of a normal HTTP request by comparing the reconstruction error between the original marker sequence and the reconstructed marker sequence includes: Calculate the BLEU score between the original tag sequence and the reconstructed tag sequence. If the BLEU score is lower than a preset threshold, the detected HTTP request is determined to be an abnormal attack request; otherwise, it is a normal request.
8. The method according to claim 7, characterized in that, The preset threshold is adjusted based on the false alarm rate and false negative rate in actual application scenarios.