A dual-channel web attack detection method based on BERT pre-trained model

Abstract

This invention discloses a dual-channel Web attack detection method based on a BERT pre-trained model, comprising: extracting URLs, request methods, payloads, and corresponding tag information from HTTP request messages in a dataset, and performing normalization preprocessing; independently encoding URL and payload parameters using pre-trained URLBERT and SecBERT, and generating context-aware embedding representations of multiple request parameters for both URLs and payloads; fusing the embedding representations of URLs and payloads, concatenating them to obtain a joint feature vector, and inputting it into a fully connected layer for classification; training the model using a training set, optimizing the model using a cross-entropy loss function as the optimization objective, optimizing the model through a backpropagation mechanism, simultaneously co-learning URLBERT and SecBERT, and evaluating its detection performance on a test set. Using this invention, the features of request URLs and payloads are fully mined in Web attack detection tasks, the relationship between parameter combinations is modeled, and attacks can be effectively traced back to their source.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a dual-channel web attack detection method based on a BERT pre-trained model. Background Technology

[0002] In the era of big data and artificial intelligence, web applications have become a core component of modern society and business activities. Web applications store vast amounts of sensitive information, such as personal data, trade secrets, and financial information, making the protection of this data crucial. In the context of artificial intelligence, deep learning technology plays a vital role in the field of web security. Through deep learning methods, potential cyberattacks can be detected and effective solutions can be provided, helping to address increasingly complex cybersecurity threats. This paper will explore the application and research of deep learning in web attack detection in the era of big data and artificial intelligence.

[0003] Common web attacks include Structured Query Language (SQL) injection, Cross-Site Scripting (XSS) attacks, Carriage Return Line Feed (CRLF) injection, sensitive data exposure, and access control violations. As the types and complexity of web attacks continue to increase, traditional detection methods are struggling to counter these sophisticated attacks. Therefore, web security researchers are actively exploring novel attack detection methods based on deep learning.

[0004] Traditional web attack detection techniques primarily rely on blacklist / whitelist methods. This approach identifies potential cyberattacks by defining normal or abnormal rules. However, this method has limitations because attackers often find clever ways to circumvent known rules and signatures. To effectively detect web attacks, traditional machine learning detection methods have emerged, involving more comprehensive features and patterns. However, this method requires experts to manually select and extract features, which may not be effective when dealing with complex text sequences in web application security. In this context, deep learning technology offers a new solution for web attack detection. Deep learning methods extract high-level abstract features from raw data through multi-layered neural networks, demonstrating strong adaptability to complex attack behaviors.

[0005] Current deep learning-based web attack detection methods typically rely on feature extraction from the URL and payload fields of HTTP requests. However, existing research still has the following shortcomings in processing these key fields:

[0006] (1) Embedding failure due to special syntax structures of URLs and payloads. Traditional methods often segment URLs and payloads into terms by payload delimiters (such as " / ", "&") and model them using word embedding models. However, the "words" in URLs and payloads are often not standard language units, usually containing abbreviations, dense symbols, concatenated words (such as "api", "v1", "getUserInfo", etc.), or user-defined formats, which cannot be accurately matched in a limited vocabulary, thus affecting the embedding quality. In addition, the large URL vocabulary can easily cause a surge in model parameters, leading to computational and storage burdens and reducing detection efficiency.

[0007] (2) Improper handling of payload order. Parameters (payload) in HTTP requests are usually in the form of unordered key-value pairs. Payloads in different orders are functionally equivalent (e.g., "id=123&name=John" and "name=John&id=123"). However, existing methods often treat them as ordered sequences, ignoring their combination relationships. This may lead to different discrimination results for equivalent inputs by the model, reducing generalization ability.

[0008] (3) Lack of attack attribution capability. In traditional Web attack detection tasks, models often focus on determining whether a request is malicious, but rarely pay attention to where the attack originated or which parameters triggered it. While this "black box" detection improves accuracy, it lacks the ability to interpret the attack structure, making it difficult to support subsequent security responses, rule formulation, and attribution analysis.

[0009] To address key issues neglected in existing research on web attack detection, this study proposes WADBERT (WebAttackDetection based on PretrainedBERT models), a model based on pre-trained models URLBERT and SecBERT. WADBERT effectively alleviates the difficulty in representing non-standard words in URLs and payloads by introducing a character-level embedding strategy instead of traditional word segmentation methods. During payload representation, a multi-head attention mechanism is used to capture the combined relationships of unordered request parameters. Simultaneously, the multi-head attention mechanism assigns importance scores to each request parameter, enabling the model not only to identify attacks but also to locate their sources, enhancing its practical value in real-world security response and attribution analysis. Experimental results show that this method achieves an accuracy of 99.87% on the CSIC2010 public dataset, demonstrating excellent performance in web attack detection.

[0010] In existing research, character embedding-based methods have advantages in representing the unique syntactic structures of URLs and payloads, effectively capturing special symbols and non-standard lexical features. CNN-based methods are beneficial for extracting the combinational relationships between local context and parameters in the payload, demonstrating strong detection capabilities in handling web attacks. In contrast, while RNN-based methods consider the potential connections between URLs and payloads to some extent, they neglect the combinational relationships between parameters within the payload. Therefore, in web attack detection tasks, relying solely on a single-dimensional feature representation is insufficient for accurately identifying diverse attack methods. To achieve more comprehensive detection capabilities, it is necessary to comprehensively consider the special structures of URLs and payloads, the combinational relationships between payload parameters, and the potential interaction information between URLs and payloads, thereby improving the model's sensitivity to potential attacks and its judgment accuracy. Summary of the Invention

[0011] The technical problem to be solved by the embodiments of the present invention is to provide a dual-channel Web attack detection method based on a BERT pre-trained model, which can solve the problems of insufficient feature mining of request URL and payload, lack of modeling of parameter combination relationship, and lack of attack tracing ability in Web attack detection tasks.

[0012] To address the aforementioned technical problems, this invention provides a dual-channel Web attack detection method based on a BERT pre-trained model, comprising the following steps:

[0013] S1: Extract the URL, request method, payload, and corresponding tag information from the HTTP request messages in the dataset, and perform normalization preprocessing;

[0014] S2: Use pre-trained URLBERT and SecBERT to independently encode the URL and Payload parameters, and generate context-aware embedding representations of multiple request parameters of the URL and Payload respectively.

[0015] S3: The embedded representations of the URL and the payload are fused and concatenated to obtain a joint feature vector, which is then input into the fully connected layer for classification;

[0016] S4: Train the model using the training set, optimize the model using the cross-entropy loss function as the optimization objective, optimize the model using the backpropagation mechanism, simultaneously learn URLBERT and SecBERT together, and evaluate its detection performance on the test set.

[0017] Furthermore, the preprocessing in S1 includes the following steps:

[0018] The URL portion is preprocessed by performing structural normalization, standardizing capitalization, and tagging of request methods;

[0019] The preprocessing of the payload involves recursive decoding, character normalization, and key-value pair formatting.

[0020] Furthermore, the preprocessing of the payload portion also includes using a secure decoding strategy. In each decoding attempt, a custom function is used to simulate the parsing of UTF-8 byte sequences, restoring only legal characters and preserving the original encoding of illegal or incomplete bytes, thus ensuring semantic integrity and security.

[0021] Furthermore, S2 also includes the step of:

[0022] S21: The URL is modeled using a character-level embedding strategy, a character vocabulary is constructed and the corresponding URLBERT embedding matrix is ​​extracted. The URL is converted into a character sequence and then input into the Transformer encoder to extract the semantic features of the URL.

[0023] This includes the following steps:

[0024] S211: Perform character-level tokenization on the URL, parsing it into a continuous sequence of characters. Each character is treated as an independent token and mapped to a corresponding index sequence according to a predefined character vocabulary, forming an input ID vector. If a character not in the character vocabulary appears in the input URL, it is uniformly replaced with a [UNK] token. The index sequence is concatenated with the special symbols [CLS] and [SEP] to form a complete input sequence, which is then input into the URLBERT model for subsequent semantic encoding. In each training batch, the longest URL character sequence length in that batch is selected as the padding standard, and the remaining shorter sequences are padded with [PAD] tokens at the end to ensure the consistency of tensor dimensions. During Transformer encoding, the position corresponding to the [PAD] token is masked by the attention mask to ensure that it does not interfere with the contextual semantic modeling.

[0025] S212: By searching the pre-constructed character embedding matrix, the token embedding corresponding to each character ID is obtained. After obtaining the token embedding representation, position embedding and type embedding are further introduced for vector augmentation processing. Position embedding is used to identify the position of each character in the sequence, helping the model capture the order information between characters; type embedding is used to distinguish the belonging relationship between different input segments; the character embedding, position embedding and type embedding are added together to form the final input representation of the URL input sequence in the Transformer encoder.

[0026] S213: The embedding vector is fed into the URLBERT model as input for encoding. In the internal structure of URLBERT, the input sequence passes through multiple stacked Transformer encoding layers in sequence. Each encoder layer consists of a multi-head attention mechanism, a feedforward neural network, residual connections, and layer normalization. It captures the semantic dependencies and structural features within the sequence from different subspaces and finally extracts and integrates the contextual information in the character-level input.

[0027] S214: After the encoding process of the URLBERT model, the hidden state representation corresponding to each input token can be obtained, denoted as [U0, U1, U2, ... Un]. The hidden state U0 corresponding to the first special marker [CLS] serves as the global semantic representation U of the entire URL sequence. embedding ;

[0028] S22: The pre-trained model SecBERT is used to independently encode the payload request parameters in the HTTP request, generating context-aware embedding representations of the request parameters. Then, a multi-head attention mechanism is introduced to capture the combination relationship between the request parameters, and the pooling operation is used to integrate them into the semantic features of the entire payload, which serve as one of the inputs to the subsequent classification module.

[0029] This includes the following steps:

[0030] S221: Perform character-level tokenization on the payload, parsing each parameter value into a continuous sequence of characters. Each character is used as an independent token and mapped to the corresponding index sequence according to a predefined character vocabulary to form a standard input ID vector.

[0031] S222: By searching the pre-constructed character embedding matrix, the token embedding corresponding to each character is obtained. After obtaining the token embedding representation, when embedding each request parameter, position encoding is introduced to identify the relative position information of the character in the sequence; a unified type encoding is added; after adding the character embedding, position encoding and type encoding, the final input representation is formed and input into the SecBERT model for processing.

[0032] S223: After each parameter embedding sequence is input into SecBERT, it is processed by a multi-layer stacked Transformer encoder. Each encoder layer contains a multi-head attention mechanism, a feedforward neural network, and residual connections and layer normalization structures to gradually extract semantic features from the character-level input. Finally, the parameter-level [CLS] vector is extracted as its context semantic representation.

[0033] S224: All attention-enhanced parameter representations will be fused through average pooling to form a unified global semantic representation P for the payload.embedding This serves as the input for the subsequent feature fusion module.

[0034] Further, step S3 includes:

[0035] S31: Obtain the global semantic vector U of the URL embedding and the global semantic vector P of the payload embedding The two are combined into a fusion vector f using a direct concatenation method;

[0036] S32: Input the fused vector f into the fully connected neural network for classification.

[0037] Furthermore, the calculation method for using the cross-entropy loss function as the optimization objective in S4 is as follows:

[0038]

[0039] Among them, y i This represents the one-hot encoding of the real label. This represents the probability value of the i-th class predicted by the model, where K is the total number of classes.

[0040] Implementing the embodiments of this invention has the following beneficial effects: This invention addresses the shortcomings of existing methods in Web attack detection tasks regarding the special syntactic structures of URLs and payloads, the unordered combination characteristics of payload parameters, the potential correlation between URLs and payloads, and the interpretability of attack attribution. It proposes a dual-channel Web attack detection method based on pre-trained language models—WADBERT. This model is based on two pre-trained language models, URLBERT and SecBERT, extracting semantic features of URL and payload parameters respectively. It enhances the modeling ability for non-natural language structures through character-level embedding, and introduces a multi-head attention mechanism into the payload representation to identify the combination relationships between parameters and the location of the attack source. This approach balances accuracy and interpretability, constructing an attack identification framework suitable for Web security scenarios. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the overall framework structure of the present invention;

[0042] Figure 2 This is a diagram illustrating the URL processing process;

[0043] Figure 3 This is a schematic diagram illustrating the text length distribution of URLs in the CSIC2010 dataset;

[0044] Figure 4 This is a schematic diagram of the payload preprocessing process;

[0045] Figure 5 This is a diagram illustrating the length distribution of the key-value pair text in the payload;

[0046] Figure 6 This is a schematic diagram of the URL feature extraction process;

[0047] Figure 7 This is a schematic diagram of the feature extraction process of the payload; Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0049] An embodiment of the present invention provides a dual-channel web attack detection method based on a BERT pre-trained model, the main steps of which include:

[0050] In the data preprocessing stage, the URL, request method, request parameters (Payload), and corresponding tag information are extracted from the HTTP request messages in the CSIC2010 dataset. For the URL, structural normalization, lowercase standardization, and request method tokenization are performed. For the Payload, recursive decoding, character standardization, and key-value pair formatting are performed sequentially to ensure the consistency and discriminability of the model input data.

[0051] During the feature extraction phase, the model uses pre-trained URLBERT and SecBERT to analyze the URL and payload, respectively.

[0052] The parameters are encoded independently. Both employ character-level embedding to enhance the model's ability to distinguish aberration attack text. In the payload section, a multi-head attention mechanism is introduced to interactively process the contextual representation of the request parameters within the payload, thereby capturing potential combinations of request parameters.

[0053] During the model building phase, the WADBERT model integrates the embedded representations of URL and Payload based on feature extraction, concatenating them to obtain a joint feature vector, which is then input into a fully connected layer for classification. The model structure features dual-channel feature extraction, the ability to distinguish deformed URLs and Payloads, and a multi-head attention mechanism to capture the combined relationships between request parameters.

[0054] Finally, in the model training and evaluation phase, the model was trained using the training set from the CSIC2010 dataset, and its detection performance was evaluated on the test set. Accuracy, precision, recall, and F1 score were used as the main metrics to compare and analyze the performance of the WADBERT model in the Web attack detection task.

[0055] On the one hand, this invention systematically compares WADBERT with various existing detection methods (such as CNN, RNN, MRN, etc.) and traditional machine learning methods (such as KNN, SVM, XGBoost, etc.). The results show that WADBERT significantly outperforms the comparison models in accuracy, precision, recall, and F1 score, fully demonstrating the model's superior performance in Web attack detection tasks. On the other hand, this paper explores the impact of different embedding strategies, feature channel configurations, and multi-head attention mechanisms on model performance through a series of ablation experiments. Experimental results show that character-level embeddings have stronger expressive power in handling obfuscated deformations and non-natural language structures; dual-channel structures are superior to single-channel inputs, and can more completely restore attack semantics; while the attention mechanism not only improves detection accuracy but also supports the visualization and tracing of attack parameters, enhancing the model's interpretability and practicality.

[0056] Specifically, this invention uses the publicly available HTTP CSIC2010 dataset for experiments. This dataset contains 61,065 automatically generated web requests and is mainly used to test the performance of network attack protection methods. It was produced by the Institute for Information Security of the Spanish Higher Academic Research Council (Consejo Superior de Investigaciones Cientificas, CSIC). CSIC2010 contains HTTP data traffic generated for e-commerce web applications.

[0057] The CSIC2010 dataset contains 36,000 normal requests and 25,065 anomalous requests. HTTP requests are marked as normal or anomalous, and the dataset includes various attacks such as SQL injection, buffer overflow, information gathering, file disclosure, CRLF injection, XSS, server-side inclusion, and parameter tampering. This dataset is widely used in research on web detection methods both domestically and internationally.

[0058] In the CSIC2010 dataset, each HTTP request is parsed into multiple fields, including the request method, request path (URL), protocol type, request header information (such as UserAgent, Accept, Host, ContentType, etc.), and request body content (payload). Finally, a label field indicates whether it is an abnormal request. The field design follows the standard structure of the HTTP protocol, making it easy for tasks to extract relevant information as needed.

[0059] This invention extracts two key fields from HTTP requests that are highly relevant to user input: the URL (after removing the query string) in the request line, and the payload carried in the request body or query string. The former primarily identifies the target resource requested by the client, while the latter carries the specific parameter information submitted by the user. These two parts not only occupy an important position in the HTTP request but, because they are directly exposed to user input, they become the main entry point for attackers to launch web attacks. In practical applications, URLs are often used to control access to and location of resources. Attackers may bypass security policies and obtain unauthorized resources through path tampering, directory traversal, or illegal route construction, thus posing a potential internal threat. The parameters carried in the payload are more likely to become the carriers of various injection attacks, such as SQL injection, cross-site scripting (XSS), and command execution. These attacks interfere with backend logic by manipulating input content, exhibiting clear external threat characteristics. Considering the differences in structure and attack vectors between URLs and payloads, they are processed separately during feature modeling, encoded separately, and then their representations are merged to improve the model's ability to identify complex attack behaviors.

[0060] Specifically, the present invention is implemented through the following steps.

[0061] S1: Extract the URL, request method, payload, and corresponding tag information from the HTTP request messages in the dataset, and perform normalized preprocessing.

[0062] S11: Preprocessing of the URL portion.

[0063] In web attack detection tasks, URLs, as one of the core fields in HTTP requests, carry rich structural and semantic information. Attackers often construct specially formatted URLs to achieve attack methods such as directory traversal and backup reading. These attack methods typically bypass system security mechanisms by embedding specific symbols or malicious parameters in the URL path, allowing access to sensitive information or execution of malicious operations. The raw data of the CSIC2010 dataset exists in the format of HTTP requests, where each record contains complete URL information. Before formally building the model, the URLs need to undergo data analysis and cleaning to ensure the accuracy and consistency of the data.

[0064] To ensure the standardization and consistency of URL data, the original URL text was preprocessed before URL feature extraction. For example... Figure 2 As shown, the preprocessing process includes standardizing the URL path portion, unifying the capitalization, and introducing the request method to provide consistent and clear input data for subsequent text embedding, thereby ensuring the efficiency and accuracy of model training.

[0065] In the URL preprocessing process, the URL structure is first parsed to extract the protocol (scheme), hostname (host), and path portion of each URL. URL path portions often contain redundant forward slashes ( / / ), and this inconsistency can affect subsequent data analysis and modeling. Therefore, this paper normalizes the path by removing redundant forward slashes, ensuring the uniformity and standardization of the path structure, thereby reducing redundant information in the data and improving processing reliability. Secondly, to eliminate feature redundancy caused by case differences in URL paths, all URLs are converted to lowercase. This operation effectively avoids feature redundancy caused by case differences, allowing the model to focus on the core content of the URL during training without being interfered with by format differences. Finally, after cleaning and normalizing the URLs, request methods (such as GET, POST, PUT, DELETE, etc.) are further introduced and appended to the URL using the [METHOD] marker to enhance the semantic expression of HTTP requests.

[0066] HTTP request methods define a client's intent to manipulate server resources and are associated with specific types of attack behaviors. Therefore, this tokenization process preserves URL information and combines the request method with the URL, providing the model with richer contextual information. For example, a POST request URL will be represented as [POST]http: / / example.com / login, while a GET request will be represented as [GET]http: / / example.com / index. Through these preprocessing steps, URLs become more consistent and standardized, ensuring the model can better learn the semantic features of URLs and improving the accuracy and reliability of web attack detection.

[0067] Furthermore, to better understand the structural characteristics of URLs and provide a basis for setting the input length of subsequent models, a statistical analysis of the distribution of URL text length was performed. The histogram of URL text length distribution is shown below. Figure 3As shown, the URL lengths are mainly concentrated between 45 and 65 characters. Furthermore, descriptive statistical analysis of the data yields relevant statistics on URL path length. Specifically, the average URL length is 55.98 characters, with a standard deviation of 5.65. The shortest URL is 28 characters, and the longest is 93 characters. This analysis is significant for subsequent URL feature extraction. Specifically, when using language models like BERT, the length of the input text sequence usually needs to be fixed. Therefore, based on the distribution of URL text lengths, an appropriate input length can be selected, and text exceeding this length can be truncated, or shorter text can be padded, which helps ensure the consistency of input data and improve the efficiency of model training.

[0068] S12: Preprocessing of the Payload portion.

[0069] In web attack detection scenarios, malicious requests are highly diverse and obfuscated. To improve the model's ability to identify attack characteristics, the raw payload data needs to be preprocessed. For example... Figure 4 As shown, the payload's preprocessing process covers URL recursive decoding, Unicode standardization, and data format normalization, aiming to improve the model's ability to identify malicious semantic features and its generalization performance.

[0070] First, attackers use URL encoding in the form of %xx (e.g., %3C represents <) to hide key features, bypassing static rule matching and character filtering mechanisms. To improve the visibility of malicious features, this paper performs standard URL decoding on all payload data. In real-world scenarios, it was found that some attackers use multi-layered encoding techniques, such as encoding < as %3C, then encoding % as %25, ultimately forming %253C. In this case, a single decoding operation cannot completely restore the characters; therefore, a recursive decoding strategy is needed to repeatedly perform URL decoding on the encoded content in the payload until the content before and after decoding is consistent or the maximum number of decoding attempts is reached. This strategy can effectively identify attack payloads with nested encoding forms, such as %2525253Cscript%2525253E, which is gradually restored to... <script>，提高模型对混淆攻击的检测能力。此外，URL解码过程还需要对特殊字符进行处理，如%00（空字符）等。某些字符在解码后可能引发系统解析差异，因此，为防止解析歧义或引发误解码，本文采用安全解码策略。在每次解码尝试中，使用自定义函数模拟UTF-8字节序列解析，仅对合法字符进行还原，对非法或不完整的字节保留其原始编码（如%FC%80、%ZZ等），确保语义完整性与安全性。

[0071] 其次，为消除语义或视觉上的字符变体，Payload需进行Unicode标准化。不同攻击者可能使用全角字符、组合字符或不同的编码方式构造功能等价但形式不同的Payload，如全角字符"＜”与半角"<”实际语义一致但在编码上不同。为此，预处理阶段统一采用NFKC（NormalizationFormCompatibilityComposition）标准对字符进行归一化，将所有组合字符归并、全角字符转换为半角形式，并统一使用UTF-8编码，确保输入到模型的文本具有一致性表示。

[0072] 最后，为适应WADBERT模型对输入格式的要求，将原始Payload中的多个键值对统一转化为一个字符串列表，依次提取每个键值对的内容，将其格式化为独立的字符串，并按顺序加入列表中，最终形成由若干键值对组成的列表。列表格式为后续的序列编码与模型训练提供了标准化的输入形式。

[0073] 此外，为了理解Payload的结构特征，并为后续SecBERT模型的输入长度设定提供依据，对Payload中键值对的文本长度分布进行了统计分析。Payload的单个键值对文本长度的分布如图5所示，HTTP请求中传递的数据具有较强的不确定性和差异性，Payload中键值对长度在不同请求中存在显著波动。可以看出大部分长度集中在5到50个字符之间，呈现出较为稳定的结构特征。通过进一步的描述性统计分析可知，单个键值对的平均长度为15.49个字符，标准差为11.91，最短为0个字符（即请求中无参数），最长达173个字符。该分析结果为模型输入设计提供了依据，在采用BERT等预训练语言模型时，需对输入文本进行定长处理，因此可根据长度分布设定合理的截断或填充策略，以保证输入数据的一致性、减少信息损失，提升模型训练效率。

[0074] S2：使用预训练的URLBERT与SecBERT分别对URL与Payload参数进行独立编码，并分别生成URL和Payload多个请求参数的上下文感知的嵌入表示。

[0075] WADBERT（WebAttackDetectionbasedonPretrainedBERTmodels）是面向Web攻击检测任务设计的双通道语义建模框架，整体由URL特征提取模块、Payload特征提取模块以及特征融合与分类模块组成。模型以HTTP请求中的URL和Payload参数为输入，分别对其进行语义嵌入，用于识别Web请求中的异常行为模式。具体而言，WADBERT采用两个相互独立的预训练BERT语言模型，其中URL部分由引入对比学习与对抗训练机制的预训练模型URLBERT进行编码，以强化对URL结构与语义的理解；Payload的请求参数由针对网络安全领域预训练模型的SecBERT进行编码，以提升模型对恶意参数特征的识别能力。两者分别生成URL和Payload多个请求参数的上下文感知的嵌入表示，用以保留各自的语义特征。随后，为了获得Payload的整体语义特征，模型引入多头注意力机制，对多个请求参数的嵌入表示进行交互建模，融合其上下文信息，捕捉请求参数之间的组合关系，最终通过平均池化得到表征整体Payload语义的特征向量。最后，模型将URL与Payload的嵌入向量进行拼接，形成联合特征表示，并输入至全连接层后通过Softmax函数进行分类，生成攻击与非攻击类别的概率分布。在模型训练过程中，根据预测概率与真实标签之间的差异，采用交叉熵损失函数作为优化目标，对模型参数进行更新。

[0076] 具体而言，S2包括步骤：

[0077] S21：采用字符级嵌入策略对URL进行建模，构建字符词表并提取对应URLBERT的嵌入矩阵，将URL被转换为字符序列后输入至Transformer编码器提取URL的语义特征。

[0078] 在预训练模型URLBERT的基础上，URL特征提取模块采用字符级嵌入策略对URL进行建模，构建字符词表并提取对应URLBERT的嵌入矩阵。最后URL被转换为字符序列后输入至Transformer编码器提取URL的语义特征。

[0079] URLBERT是针对URL分类任务设计的BERT架构预训练模型，超参数如表1所示。URLBERT通过自监督对比学习和虚拟对抗训练，增强了模型对URL结构变化的识别能力与抗扰动性能。然而，WordPiece子词级分词方法在处理URL时仍存在一定局限，尤其在Web安全场景中，攻击者经常通过微小字符级扰动（如插入符号、拼接变形、缩写变形）绕过检测，这些细粒度变化可能在子词切分中被掩盖，导致语义信息损失。

[0080]

[0081] 表1URLBERT网络结构的超参数

[0082] 本发明对原始URLBERT架构进行了适配性修改，将其分词机制替换为字符级（character-level）嵌入策略。该方式以单字符为最小建模单元，不依赖固定词表，能够更细致地表达URL中的符号结构、路径片段及可疑模式，提升模型对恶意URL结构的鲁棒性和泛化能力。

[0083] 具体而言，首先对数据集中的全部预处理后的URL样本进行了字符统计，结果显示共包含66种不同字符，包括26个英文字母（a–z）、10个阿拉伯数字（0–9），以及30个常见的特殊符号，如斜杠（ / ）、问号（?）、等号（=）、点号（.）、百分号（%）、连接符（-）、下划线（_）、井号（#）等。这些字符构成了模型用于嵌入的基础字符表。此外，为兼容BERT架构的输入需求，词表中还额外加入了5个特殊标记符：[PAD]（用于补全序列）、[UNK]（表示未知字符）、[CLS]（用于提取全局表示）、[SEP]（分隔标记）和[MASK]（用于预训练任务），最终构建的字符词表共包含71个符号。随后，基于预训练模型URLBERT的嵌入层提取相应的字符向量构建嵌入矩阵，如算法1所示。因为所有字符均在URLBERT的词表中具有对应项，因此无需额外的随机初始化步骤，从而保证了字符嵌入的一致性与预训练URLBERT表征能力的继承。最终构建的字符嵌入矩阵可与URLBERT编码器部分对接，实现字符级嵌入的URL特征提取。

[0084] URL的特征提取过程如图6所示：首先对URL执行字符级标记化（tokenization）操作。具体而言，URL首先被解析为连续的字符序列，每个字符作为独立的Token，根据预定义的字符词表映射为对应的索引序列，形成输入ID向量。若输入URL中出现不在字符词表中的字符（理论上不会出现，但仍保留鲁棒性处理逻辑），则统一替换为[UNK]标记。最终，该索引序列将与[CLS]和[SEP]特殊符号拼接，构成完整的输入序列，并输入到URLBERT模型进行后续的语义编码。在具体实现过程中，为提升模型的计算效率与批处理能力，输入序列采用了批次内最大长度（max-lengthwithinbatch）填充策略进行对齐。由于对数据集中URL文本长度的统计分析显示，其实际最大长度远低于URLBERT支持的最大输入长度，因此无需统一截断至固定上限，从而避免了资源浪费。具体而言，在每个训练批次中，选取该批次中最长的URL字符序列长度作为填充标准，其余较短序列在末尾补齐[PAD]标记，保证张量维度的一致性。此外，在Transformer编码过程中，[PAD]标记所对应的位置会被attentionmask屏蔽，确保其不会对上下文语义建模产生干扰。

[0085] 随后，将输入ID向量映射至对应的嵌入向量，即通过查找预构建的字符嵌入矩阵，获取每个字符ID对应的token嵌入。在获得标记嵌入表示后，进一步引入位置嵌入（PositionalEmbedding）与类型嵌入（SegmentEmbedding）进行向量增强处理。其中，位置编码用于标识每个字符在序列中的位置，帮助模型捕捉字符间的顺序信息；类型编码则用于区分不同输入片段（如句子对）之间的归属关系。由于输入仅包含单一URL序列，因此所有位置的类型编码均统一设为0。最终，字符嵌入、位置嵌入与类型嵌入三者相加，形成URL输入序列在Transformer编码器中的最终输入表示。

[0086] 接着，嵌入向量作为输入被送入URLBERT模型进行编码。在URLBERT的内部结构中，输入序列依次通过多个堆叠的Transformer编码层。每一层编码器均由多头注意力机制、前馈神经网络和残差连接与层归一化组成，能够从不同子空间捕捉序列内部的语义依赖关系和结构特征。随着Transformer编码层的逐层堆叠，模型逐步提取和整合字符级输入中的上下文信息，生成更加丰富、全局化的表示向量。

[0087] 最终，经过URLBERT模型的编码过程，可获得每个输入token对应的隐藏状态表示，记作[U0,U1,U2,...Un]。特别地，第一个特殊标记[CLS]对应的隐藏状态U0，作为整个URL序列的全局语义表示Uembedding，用于后续的分类任务。

[0088] S22：使用预训练模型SecBERT对HTTP请求中的Payload请求参数独立编码，生成请求参数上下文感知的嵌入表示，随后引入多头注意力机制捕捉请求参数之间的组合关系，并通过池化操作整合为Payload整体的语义特征，作为后续分类模块的输入之一。

[0089] Payload是HTTP请求中用于承载用户输入的关键部分，通常以多个键值对（key=value）的形式存在，攻击者常通过在参数中注入恶意代码（如SQL语句、脚本片段或命令字符串）实施Web攻击，Payload因而成为Web攻击检测中最直接的风险来源。为了获取Payload的整体语义特征，WADBERT从参数级别出发，对各个参数进行逐项表示，并进一步捕获其组合关系，以增强模型对攻击的识别能力。

[0090] 与URLBERT相同，SecBEER默认采用WordPiece子词嵌入方式对输入文本进行嵌入。

[0091] 然而，Payload通常缺乏明确的语法结构，且攻击者常通过字符级扰动（如如符号拼接、空格注入、变形重排等）规避检测，传统的子词切分方式容易掩盖微观攻击特征，导致模型难以识别关键模式。为提升对恶意参数的建模能力，本文将SecBERT的输入嵌入机制替换为字符级嵌入策略，即以单字符为最小建模单元，重构嵌入表示。字符嵌入矩阵的构建方法与URL模块中一致，通过预定义的字符词表（共100个字符，额外加入了5个特殊标记符[PAD]、[UNK]、[CLS]、[SEP]和[MASK]）从原始嵌入层中提取对应向量，生成字符嵌入矩阵，用于对Payload字符序列进行初始表示。此外，SecBERT同样基于BERT架构，其模型结构及主要超参数设置与URLBERT保持一致，仅在词表和嵌入层的权重上有所差异，词表为100个字符的词表，嵌入层的权重修改为100个字符对应原始嵌入层的嵌入矩阵，为了更好地适应Web安全领域的语料特性与任务需求。

[0092] 模型提取payload的特征流程如图7所示，首先对Payload执行字符级标记化（tokenization）操作，将每个参数值被解析为连续的字符序列，每个字符作为独立的Token，根据预定义的字符词表映射为对应的索引序列，构成标准的输入ID向量。若输入字符不在词表中（实际中不会出现，仍保留鲁棒性处理逻辑），统一映射为[UNK]标记。每个参数的索引序列均添加[CLS]和[SEP]特殊标记，用于构建完整输入序列，并分别送入SecBERT编码器处理。在具体实现过程中，模型采用批次内最大长度（max-lengthwithinbatch）填充策略提升并行计算效率。即在每个训练批次中，选取该批次中最长的key=value字符序列长度作为统一对齐标准，其余较短序列末尾补齐[PAD]标记，保证张量维度一致。在Transformer编码阶段，这些[PAD]位置通过attentionmask进行屏蔽，不会对上下文建模产生影响。

[0093] 随后，将输入序列中的字符标记映射至对应的嵌入向量，即通过查找预构建的字符嵌入矩阵，获取每个字符对应的token嵌入。在获得标记嵌入表示后，在对每个请求参数进行嵌入时，引入位置编码（PositionalEmbedding），用于标识字符在序列中的相对位置信息；同时，为区分不同句子片段，添加统一的类型编码（SegmentEmbedding）。由于输入仅包含单个参数序列，类型编码全部置为0。字符嵌入、位置编码与类型编码相加后，构成最终输入表示，并输入至SecBERT模型进行处理。

[0094] 每个参数嵌入序列在输入至SecBERT后，经过多层堆叠的Transformer编码器处理。每层编码器均包含多头注意力机制、前馈神经网络及残差连接与层归一化结构，逐步从字符级输入中提取语义特征。最终提取出参数级的[CLS]向量作为其上下文语义表示。

[0095] 对于一个HTTP请求中的多个参数，模型将其分别编码后的[CLS]向量组合为一个参数嵌入集合[P1,P2,P3,...Pn]，n为请求参数的个数。由于请求参数在语义上本质为一个无序集合，传统的序列建模方式难以刻画参数之间存在的协同或交互关系。为此，WADBERT引入多头注意力机制对该嵌入集合进行交互处理，旨在捕捉参数之间的组合关系。具体而言，多头注意力机制通过计算任意参数对之间的相关性，自适应地学习不同参数间的语义依赖。例如，在典型的组合式SQL注入中，攻击者可能将逻辑判断语句拆解到不同的参数中，例如search=admin与filter='OR1=1--，其中前者指定目标关键词，后者则注入绕过查询条件的语句结构。对于此类跨参数组合攻击，模型仅依靠单个参数特征难以识别其完整逻辑，因此有必要构建全局上下文关联以还原攻击语义。

[0096] 多头注意力机制通过多个独立的子注意力头，从不同的表示子空间评估参数间的交互强度。每个注意力头将根据参数间的语义关联动态赋予权重，从而模拟攻击请求参数之间存在的组合关系。通过多头注意力机制机制，模型不仅能够识别出潜在的组合关系，还能够自动聚焦于在攻击语义中起关键作用的核心参数，如包含逻辑控制、命令入口或绕过触发点的字段。注意力分布赋予了模型一定的可解释性，即通过查看各参数的注意力得分，能够反推出当前模型认为最具攻击可能性的参数位置。这种能力对实际部署中的攻击分析与响应具有重要价值，为安全系统提供结构化的攻击溯源线索。

[0097] 最终，所有经过注意力机制强化的参数表示将通过平均池化（AveragePooling）操作进行融合，形成统一的Payload全局语义表示Pembedding，作为后续特征融合模块的输入。

[0098] S3：对URL与Payload的嵌入表示进行融合，并拼接得到联合特征向量，输入至全连接层进行分类。

[0099] 在完成URL与Payload特征的提取后，WADBERT模型将两者的语义表示进行拼接融合，形成HTTP请求特征联合表示，并通过分类模块完成最终的攻击判定任务。

[0100] 该过程有效整合了URL信息与Payload参数内容中的潜在攻击特征，增强了模型对HTTP请求整体语义的感知能力。

[0101] 具体而言，模型分别获得了URL的全局语义向量Uembedding和Payload的全局语义向量Pembedding，二者分别来自URL与Payload的特征提取模块的输出，代表了URL与Payload内部请求参数的上下文嵌入。特征融合阶段采用直接拼接方式，将两者组合为联合表示向量，计算方式为：

[0102]

[0103] 其中，[;]表示向量拼接操作，融合后的向量f同时包含URL与输Payload的语义信息，作为后续分类模块的输入。

[0104] 随后，融合后的向量f被输入至全连接神经网络进行分类判断。该分类器由一层线性层组成，输入维度为hidden_size×2即768×2=1536，最后一层输出维度为2，分别对应"攻击”与"非攻击”两个类别，输出层使用Softmax激活函数生成预测概率分布，得到Web攻击的检测结果。计算方式为：

[0105]

[0106] 其中，W和b分别为分类器的权重矩阵和偏置项，f为融合后的特征向量，表示模型预测的类别概率分布。该公式实现了将融合后的请求表示映射至固定类别空间，并通过Softmax函数输出属于每一类的预测概率。

[0107] S4：使用训练集对模型进行训练，通过交叉熵损失函数作为优化目标，通过反向传播机制对模型进行优化，同步对URLBERT与SecBERT进行协同学习，并在测试集上评估其检测性能。

[0108] 在模型训练阶段，模型采用交叉熵损失函数作为优化目标，计算方式为：

[0109]

[0110] 其中，yi表示真实标签的独热编码，表示模型预测的第i类的概率值，K为类别总数。在本研究的二分类任务中，K=2，分别对应"攻击”与"非攻击”两个类别。通过最小化该损失函数，模型能够优化网络权重，提升预测准确性。

[0111] 通过反向传播机制对整个模型进行优化，使得URL与Payload两部分特征提取模块能够协同学习，提升Web攻击检测的性能。

[0112] 本发明重点解决了以下四个方面的不足：

[0113] 第一，针对URL及Payload的特殊语法结构问题，WADBERT引入字符级嵌入策略替代传统的分词方法，通过URLBERT和SecBERT提取其字符层级的上下文信息，有效增强了模型对URL及Payload非常规表达形式的识别能力。

[0114] 第二，针对Payload参数的无序性问题，WADBERT在字符级嵌入的基础上引入多头注意力机制，以挖掘请求参数之间的潜在组合关系，提升了对跨字段攻击模式的捕捉能力。

[0115] 第三，针对URL与Payload之间缺乏关联建模的问题，WADBERT采用双通道编码策略，并在高层语义空间中进行特征融合，有效融合URL与Payload的信息，提升模型对攻击请求的识别能力。

[0116] 第四，在模型可解释性的方面，WADBERT利用注意力机制输出的权重得分，实现对各请求参数在攻击判定中的贡献度评估，具备一定程度的攻击溯源能力，有助于实际部署中的日志审计与安全响应。

[0117] 在CSIC2010数据集上的实验表明，WADBERT在准确率（99.87%）、精确率（99.98%）、召回率（99.70%）与F1分数（99.84%）等关键指标上均优于现有对比模型，验证了所提方法在Web攻击检测场景下的有效性。此外，消融实验进一步证明了字符级嵌入、多头注意力机制和双通道融合结构是模型性能提升的关键因素。

[0118] 以上所揭露的仅为本发明一种较佳实施例而已，当然不能以此来限定本发明之权利范围，因此依本发明权利要求所作的等同变化，仍属本发明所涵盖的范围。< / script>

Claims

1. A dual-channel Web attack detection method based on a BERT pre-trained model, characterized in that, Includes the following steps: S1: Extract the URL, request method, payload, and corresponding tag information from the HTTP request messages in the dataset, and perform normalization preprocessing; S2: Use pre-trained URLBERT and SecBERT to independently encode the URL and Payload parameters, and generate context-aware embedding representations of multiple request parameters for both the URL and Payload, including the following steps: S21: The URL is modeled using a character-level embedding strategy, a character vocabulary is constructed and the corresponding URLBERT embedding matrix is ​​extracted. The URL is then converted into a character sequence and input into the Transformer encoder to extract the semantic features of the URL. This includes the following steps: S211: Perform character-level tokenization on the URL, parsing it into a continuous sequence of characters. Each character is treated as an independent token and mapped to a corresponding index sequence according to a predefined character vocabulary, forming an input ID vector. If a character not in the character vocabulary appears in the input URL, it is uniformly replaced with a [UNK] token. The index sequence is concatenated with the special symbols [CLS] and [SEP] to form a complete input sequence, which is then input into the URLBERT model for subsequent semantic encoding. In each training batch, the longest URL character sequence length in that batch is selected as the padding standard, and the remaining shorter sequences are padded with [PAD] tokens at the end to ensure the consistency of tensor dimensions. During Transformer encoding, the position corresponding to the [PAD] token is masked by the attention mask to ensure that it does not interfere with the contextual semantic modeling. S212: By searching the pre-constructed character embedding matrix, the token embedding corresponding to each character ID is obtained. After obtaining the token embedding representation, position embedding and type embedding are further introduced for vector augmentation processing. Position embedding is used to identify the position of each character in the sequence, helping the model capture the order information between characters; type embedding is used to distinguish the belonging relationship between different input segments; the character embedding, position embedding and type embedding are added together to form the final input representation of the URL input sequence in the Transformer encoder. S213: The embedding vector is fed into the URLBERT model as input for encoding. In the internal structure of URLBERT, the input sequence passes through multiple stacked Transformer encoding layers in sequence. Each encoder layer consists of a multi-head attention mechanism, a feedforward neural network, residual connections, and layer normalization. It captures the semantic dependencies and structural features within the sequence from different subspaces and finally extracts and integrates the contextual information in the character-level input. S214: After the encoding process of the URLBERT model, the hidden state representation corresponding to each input token can be obtained, denoted as [U0, U1, U2, ... Un]. The hidden state U0 corresponding to the first special marker [CLS] serves as the global semantic representation U of the entire URL sequence. embedding ; S22: The pre-trained model SecBERT is used to independently encode the payload request parameters in the HTTP request, generating context-aware embedding representations of the request parameters. Then, a multi-head attention mechanism is introduced to capture the combination relationship between the request parameters, and the pooling operation is used to integrate them into the semantic features of the entire payload, which serve as one of the inputs to the subsequent classification module. This includes the following steps: S221: Perform character-level tokenization on the payload, parsing each parameter value into a continuous sequence of characters. Each character is used as an independent token and mapped to the corresponding index sequence according to a predefined character vocabulary to form a standard input ID vector. S222: By searching the pre-constructed character embedding matrix, the token embedding corresponding to each character is obtained. After obtaining the token embedding representation, when embedding each request parameter, position encoding is introduced to identify the relative position information of the character in the sequence; a unified type encoding is added; after adding the character embedding, position encoding and type encoding, the final input representation is formed and input into the SecBERT model for processing. S223: After each parameter embedding sequence is input into SecBERT, it is processed by a multi-layer stacked Transformer encoder. Each encoder layer contains a multi-head attention mechanism, a feedforward neural network, and residual connections and layer normalization structures to gradually extract semantic features from the character-level input. Finally, the parameter-level [CLS] vector is extracted as its context semantic representation. S224: All attention-enhanced parameter representations will be fused through average pooling to form a unified global semantic representation P for the payload. embedding This serves as the input for the subsequent feature fusion module; S3: The embedded representations of the URL and the payload are fused and concatenated to obtain a joint feature vector, which is then input into the fully connected layer for classification; S4: Train the model using the training set, optimize the model using the cross-entropy loss function as the optimization objective, optimize the model using the backpropagation mechanism, simultaneously learn URLBERT and SecBERT together, and evaluate its detection performance on the test set.

2. The dual-channel Web attack detection method based on the BERT pre-trained model according to claim 1, characterized in that, The preprocessing in S1 includes the following steps: The URL portion is preprocessed by performing structural normalization, standardizing capitalization, and tagging of request methods; The preprocessing of the payload involves recursive decoding, character normalization, and key-value pair formatting.

3. The dual-channel Web attack detection method based on the BERT pre-trained model according to claim 2, characterized in that, The preprocessing of the payload also includes using a secure decoding strategy. In each decoding attempt, a custom function is used to simulate the parsing of UTF-8 byte sequences, restoring only legal characters and preserving the original encoding of illegal or incomplete bytes to ensure semantic integrity and security.

4. The dual-channel Web attack detection method based on the BERT pre-trained model according to claim 1, characterized in that, S3 includes the following steps: S31: Obtain the global semantic vector U of the URL embedding and the global semantic vector P of the payload embedding The two are combined into a fusion vector f using a direct concatenation method; S32: Input the fused vector f into the fully connected neural network for classification.

5. The dual-channel Web attack detection method based on the BERT pre-trained model according to claim 1, characterized in that, The calculation method in S4, which uses the cross-entropy loss function as the optimization objective, is as follows: Among them, y i This represents the one-hot encoding of the real label. This represents the probability value of the i-th class predicted by the model, where K is the total number of classes.

Images