AI-based methods and systems for detecting and defending against web application attacks

Abstract

This disclosure presents an AI-based method and system for detecting and defending against web application attacks. The method includes: extracting the minimum effective feature set of attack behavior from historical web attack traffic data, clustering it to generate attack micro-feature clusters, and constructing an attack micro-feature cluster library; extracting key features from the access traffic of web applications, and statistically analyzing the current occurrence frequency, cumulative occurrence frequency, and combination methods of the key features; matching the key features based on the attack micro-feature cluster library to obtain matching results, and combining the statistical data to construct a structured numerical matrix and inputting it into a detection model for detection; when a malicious attack is detected, a blocking operation is performed; when the malicious attack is a new and unknown attack, its features are updated in the attack micro-feature cluster library. This method can significantly reduce computational power consumption, improve detection efficiency to adapt to high-concurrency scenarios, accurately identify deformed and fragmented new attacks, and achieve self-updating of the feature library, solving the problem of rule lag.

Description

Technical Field

[0001] This disclosure relates to the field of cybersecurity technology, and in particular to an AI-based method and system for detecting and defending against web application attacks. Background Technology

[0002] With the development of Internet technology, Web applications have become the core carrier of various businesses. Malicious attack methods such as SQL injection and cross-site scripting are becoming increasingly fragmented and mutated, and zero-day attacks are frequent. Traditional defense methods are difficult to cope with dynamically evolving attack behaviors. AI-based Web attack detection and defense technology, with its self-learning and real-time analysis capabilities, has become the core research and application direction for Web application security protection.

[0003] Existing AI-based web attack detection technologies either use deep learning to perform in-depth analysis of all traffic information, which consumes a lot of computing power, has high detection latency, and is poorly adapted to high-concurrency scenarios, or rely on coarse-grained attack feature database matching, which can only identify known fixed feature attacks and fails to identify new, modified, and fragmented attacks. Furthermore, feature database updates depend on manual intervention, resulting in rule lag. Summary of the Invention

[0004] In view of this, the present disclosure provides an AI-based method and system for detecting and defending against web application attacks, which can significantly reduce computing power consumption, improve detection efficiency to adapt to high-concurrency scenarios, accurately identify new types of attacks that are deformed and fragmented, and also realize self-updating of the feature library to solve the problem of rule lag.

[0005] In a first aspect, the embodiments of this disclosure provide an AI-based method for detecting and defending against web application attacks, employing the following technical solution:

[0006] Obtain historical web attack traffic data and extract the minimum effective set of attack behavior features from the historical web attack traffic data;

[0007] Cluster the minimum effective feature set to generate attack micro-feature clusters and construct an attack micro-feature cluster library;

[0008] Collect access traffic of web applications, extract key features of the access traffic, and count the current occurrence count, cumulative occurrence count, and combination methods of the key features;

[0009] Based on the attack micro-feature cluster library, the key features are matched using attack micro-feature cluster matching to obtain matching results;

[0010] Based on the matching results, the current occurrence count, cumulative occurrence count, and combination method of the key features, a structured numerical matrix is ​​constructed, and the structured numerical matrix is ​​input into the detection model;

[0011] When the detection model outputs a result indicating a malicious attack, a blocking operation is performed;

[0012] When the malicious attack is identified as a novel unknown attack, the features of the novel unknown attack are updated to the attack micro-feature cluster library through an incremental clustering algorithm.

[0013] Optionally, the step of acquiring historical web attack traffic data and extracting the minimum effective feature set of attack behavior from the historical web attack traffic data includes:

[0014] The acquired historical web attack traffic data is preprocessed to obtain structured core traffic data;

[0015] Extract all character fragments of key payload regions from the core traffic data of similar attacks as candidate features, and group and label the candidate features of the same type according to the attack type.

[0016] Extract common core features from the candidate features of each group of attacks of the same type, split the common core features into sub-features of the smallest character, and obtain the information entropy weight of the sub-features.

[0017] Based on the information entropy weight, the minimum effective feature is selected from the sub-features, and the selected minimum effective features constitute the minimum effective feature set.

[0018] Optionally, the step of clustering the minimum effective feature set to generate attack micro-feature clusters and constructing an attack micro-feature cluster library includes:

[0019] A two-layer semantic labeling is performed on all minimum effective features within the minimum effective feature set. The first layer of semantic labeling is the attack category label, and the second layer of semantic labeling is the attack semantic sub-label.

[0020] The smallest effective features of the same attack category label are grouped into candidate clusters;

[0021] Based on the attack semantic sub-label, obtain the weighted similarity between the minimum effective features within the candidate cluster;

[0022] Based on the weighted similarity, the candidate clusters are clustered or split to generate attack micro-feature clusters;

[0023] The generated attack micro-feature clusters are used to construct an attack micro-feature cluster library.

[0024] Optionally, obtaining the weighted similarity between the minimum effective features within the candidate cluster based on the attack semantic sub-label includes:

[0025] Obtain the character similarity, the matching degree of attack semantic sub-labels, and the association similarity of information entropy weights among the minimum effective features within the candidate cluster;

[0026] The weighted sum of the character similarity, the matching degree, and the association similarity is obtained to obtain the weighted similarity between the smallest effective features within the candidate cluster.

[0027] Optionally, the step of statistically analyzing the current occurrence count, cumulative occurrence count, and combination method of the key feature includes:

[0028] Count the number of times the key feature appears in the current access traffic;

[0029] The cumulative number of times the key feature appears in all access traffic within a preset time window is counted.

[0030] Obtain the embedding position and splicing order of the key features in the current access traffic, as well as the attack semantic logical relationship between each key feature;

[0031] The embedding location, splicing order, and attack semantic logic are associated and integrated to form the combination method of the key features.

[0032] Optionally, the AI-based web application attack detection and defense method further includes:

[0033] After clustering or splitting the candidate clusters to generate attack micro-feature clusters, the attack category labels contained in the clustered or split candidate clusters are decomposed into two-layer cluster labels.

[0034] The dual-layer cluster label is used as the exclusive cluster label for the corresponding attack micro-feature cluster;

[0035] When the candidate cluster is directly used as the attack micro-feature cluster, the attack category label contained in the candidate cluster is used as the exclusive cluster label of the corresponding attack micro-feature cluster.

[0036] Optionally, the step of performing attack micro-feature cluster matching on the key features based on the attack micro-feature cluster library to obtain matching results includes:

[0037] The smallest effective target feature with the highest similarity value to the key feature is selected from the attack micro-feature cluster library;

[0038] The matching result is composed of the key features, the target minimum effective features, the similarity value, the attack semantic sub-labels of the target minimum effective features, and the exclusive cluster labels of the attack micro-feature clusters to which the target minimum effective features belong.

[0039] Optionally, constructing a structured numerical matrix based on the matching results, the current occurrence count, cumulative occurrence count, and combination method of the key features includes:

[0040] Convert the non-numeric fields in the matching results and the combination methods into fixed encoded values;

[0041] The numerical fields in the current occurrence count, the cumulative occurrence count, the matching result, and the combination method are uniformly normalized into standardized values;

[0042] A structured numerical matrix is ​​constructed based on the fixed encoded value and the standardized numerical value.

[0043] Optionally, the detection model includes an input layer, a feature extraction layer, a classification decision layer, and an output layer;

[0044] The input layer is used to convert the structured numerical matrix into tensor data;

[0045] The feature extraction layer is used to mine multi-dimensional feature association patterns in the tensor data and generate high-dimensional feature vectors.

[0046] The classification decision layer is used to obtain the probability distribution of two types of labels, malicious attack and normal traffic, corresponding to the high-dimensional feature vector, and output the malicious attack judgment result based on the probability distribution.

[0047] The output layer is used to convert the malicious attack judgment result into a standardized format and output the detection result.

[0048] Secondly, this disclosure also provides an AI-based web application attack detection and defense system, employing the following technical solution:

[0049] The extraction module is used to acquire historical web attack traffic data and extract the minimum effective feature set of attack behavior from the historical web attack traffic data.

[0050] The clustering module is used to cluster the minimum effective feature set, generate attack micro-feature clusters, and construct an attack micro-feature cluster library;

[0051] The statistics module is used to collect access traffic of the web application, extract key features of the access traffic, and count the current occurrence count, cumulative occurrence count, and combination method of the key features;

[0052] The matching module is used to perform attack micro-feature cluster matching on the key features based on the attack micro-feature cluster library to obtain the matching result;

[0053] The construction module is used to construct a structured numerical matrix based on the matching results, the current occurrence count, cumulative occurrence count, and combination method of the key features, and input the structured numerical matrix into the detection model;

[0054] The execution module is used to perform a blocking operation when the detection model outputs a detection result of a malicious attack;

[0055] The update module is used to update the features of the new unknown attack to the attack micro-feature cluster library by using an incremental clustering algorithm when the malicious attack is identified as a new unknown attack.

[0056] Thirdly, this disclosure also provides a computer device, which adopts the following technical solution:

[0057] The computer device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described AI-based web application attack detection and defense methods.

[0058] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer instructions for causing a computer to execute any of the AI-based Web application attack detection and defense methods described above.

[0059] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0060] The AI-based Web application attack detection and defense method disclosed in this embodiment first extracts the minimum effective feature set of attack behavior from historical Web attack traffic and clusters it to generate an attack micro-feature cluster library. This not only eliminates redundant features through the minimum effective feature set, significantly improving the efficiency and accuracy of feature processing, but also achieves structured and systematic storage of attack features through the clustered attack micro-feature cluster library. This establishes a standardized and targeted feature reference system for subsequent attack matching, making the initial feature foundation for attack detection more closely aligned with the core feature patterns of actual attack behavior. Simultaneously, after collecting access traffic, key features are extracted and their current occurrence frequency, cumulative occurrence frequency, and combination patterns are statistically analyzed. This allows for multi-dimensional capture of the dynamic changes and correlation patterns of traffic features, overcoming the limitations of single-feature detection. Subsequent attack judgments can combine the frequency of feature occurrence and combination patterns to more comprehensively reflect the behavioral characteristics of the traffic, providing rich and valuable feature dimensions to support accurate detection. After feature matching is completed based on the attack micro-feature cluster library, a structured numerical matrix is ​​constructed by combining the matching results with the frequency and combination information of the features and input into the detection model. This transforms multi-dimensional feature information into structured data that the model can recognize, effectively improving the detection model's accuracy and response efficiency in identifying attack behaviors and ensuring that the detection results closely match the actual attack behavior characteristics. When malicious attacks are detected, timely blocking operations are performed to quickly intercept attack behaviors, prevent web applications from being compromised, and achieve efficient defense against known attacks. When new and unknown attacks are identified, their features are updated to the attack micro-feature cluster library through an incremental clustering algorithm. This eliminates the need to re-cluster all features, significantly reducing the cost and time of feature library updates. At the same time, the coverage of the attack micro-feature cluster library is continuously enriched, giving the method self-learning and self-evolution capabilities. This enables the method to continuously improve its detection and defense capabilities against new and unknown web application attacks, achieving full-cycle and dynamic protection against web application attacks.

[0061] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1A flowchart illustrating the AI-based web application attack detection and defense method provided in this embodiment of the disclosure;

[0064] Figure 2 A flowchart illustrating the method for obtaining the minimum effective feature set provided in this embodiment of the disclosure;

[0065] Figure 3 A flowchart illustrating the method for constructing an attack micro-feature cluster library provided in this embodiment of the disclosure;

[0066] Figure 4 A flowchart illustrating the statistical method for key feature-related parameters provided in the embodiments of this disclosure;

[0067] Figure 5 A flowchart illustrating the cluster tag acquisition method provided in this embodiment of the disclosure;

[0068] Figure 6 A flowchart illustrating the structured numerical matrix construction method provided in this embodiment of the disclosure;

[0069] Figure 7 A schematic diagram of the principle of the AI-based Web application attack detection and defense system provided in the embodiments of this disclosure;

[0070] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. Detailed Implementation

[0071] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0072] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0073] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0074] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0075] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0076] Reference Figure 1 This disclosure provides an AI-based method for detecting and defending against web application attacks, including the following steps:

[0077] S1: Obtain historical web attack traffic data and extract the minimum effective feature set of attack behavior from the historical web attack traffic data;

[0078] S2: Cluster the minimum effective feature set to generate attack micro-feature clusters and build an attack micro-feature cluster library;

[0079] S3: Collect access traffic of web applications, extract key features of access traffic, and count the current occurrence, cumulative occurrence, and combination of key features;

[0080] S4: Based on the attack micro-feature cluster library, perform attack micro-feature cluster matching on key features to obtain matching results;

[0081] S5: Based on the matching results, the current occurrence count, cumulative occurrence count, and combination method of key features, construct a structured numerical matrix and input the structured numerical matrix into the detection model;

[0082] S6: When the detection model outputs the detection result of malicious attack, execute the blocking operation;

[0083] S7: When a malicious attack is identified as a new type of unknown attack, the features of the new unknown attack are updated to the attack micro-feature cluster library through an incremental clustering algorithm.

[0084] The AI-based Web application attack detection and defense method disclosed herein first extracts the minimum effective feature set of attack behavior from historical Web attack traffic and clusters it to generate an attack micro-feature cluster library. This method not only eliminates redundant features through the minimum effective feature set, significantly improving the efficiency and accuracy of feature processing, but also achieves structured and systematic storage of attack features through the clustered attack micro-feature cluster library. This establishes a standardized and targeted feature reference system for subsequent attack matching, making the initial feature foundation for attack detection more closely aligned with the core characteristic patterns of actual attack behavior. Simultaneously, after collecting access traffic, key features are extracted and their current occurrence frequency, cumulative occurrence frequency, and combination patterns are statistically analyzed. This allows for multi-dimensional capture of the dynamic changes and correlation patterns of traffic features, overcoming the limitations of single-feature detection. Subsequent attack judgments can combine the frequency of feature occurrence and combination patterns to more comprehensively reflect the behavioral characteristics of traffic, providing rich and valuable feature dimensions to support accurate detection.

[0085] After feature matching is completed based on the attack micro-feature cluster library, a structured numerical matrix is ​​constructed by combining the matching results with the frequency and combination information of the features and input into the detection model. This transforms multi-dimensional feature information into structured data that the model can recognize, effectively improving the detection model's accuracy and response efficiency in identifying attack behaviors and ensuring that the detection results closely match the actual attack behavior characteristics. When malicious attacks are detected, timely blocking operations are performed to quickly intercept attack behaviors, prevent web applications from being compromised, and achieve efficient defense against known attacks. When new and unknown attacks are identified, their features are updated to the attack micro-feature cluster library through an incremental clustering algorithm. This eliminates the need to re-cluster all features, significantly reducing the cost and time of feature library updates. At the same time, the coverage of the attack micro-feature cluster library is continuously enriched, giving the method self-learning and self-evolution capabilities. This enables the method to continuously improve its detection and defense capabilities against new and unknown web application attacks, achieving full-cycle and dynamic protection against web application attacks.

[0086] In S1, refer to Figure 2 The flowchart illustrating the method for obtaining the minimum effective feature set, "obtaining historical web attack traffic data and extracting the minimum effective feature set of attack behavior from the historical web attack traffic data," includes the following steps:

[0087] S11: Preprocess the acquired historical web attack traffic data to obtain structured core traffic data;

[0088] S12: Extract all character fragments of key payload regions from the core traffic data of similar attacks as candidate features, and complete the grouping and labeling of similar candidate features according to attack type;

[0089] S13: Extract common core features from the candidate features of each group of attacks of the same type, break down the common core features into sub-features of the smallest character, and obtain the information entropy weight of the sub-features.

[0090] S14: Based on the information entropy weight, select the minimum effective feature from the sub-features, and form the minimum effective feature set by selecting the minimum effective features.

[0091] In S11, historical web attack traffic data (including typical attack traffic such as SQL injection and cross-site scripting) is collected and aggregated through web application logs, security device alarm records, and publicly available attack traffic datasets. The collected traffic data undergoes preprocessing, including data cleaning and protocol decoding. Data cleaning involves removing redundant fields in request headers (such as User-Agent and non-core fields like Cookie), invalid null values, and duplicate traffic samples, retaining only core fields strongly related to the attack behavior (URL, request method, POST body, parameter key-value pairs, etc.). Subsequently, the retained core fields undergo protocol decoding (such as URL encoding and Base64 encoding reverse parsing), unifying character encoding formats (such as converting to UTF-8) and standardizing data structures (such as organizing by the dimensions of "attack type-request URL-parameters-payload"), ultimately obtaining structured core traffic data, ensuring that the data focuses only on key areas related to the attack behavior without redundant interference.

[0092] In S12, based on the obtained core traffic data, for different attack types such as SQL injection and cross-site scripting (XSS), the corresponding key payload areas are located (e.g., SQL injection focuses on URL parameter values ​​and query segments in the POST body, while XSS focuses on form submission content and script fragments in the URL suffix). By combining regular expression matching and character slicing, all consecutive character segments within each key payload area are extracted as candidate features (e.g., union select for SQL injection traffic). Extract all non-empty character fragments such as union, select, union select, etc. from the users payload. After completing the candidate feature extraction, group the candidate features of the same type according to the attack type (such as SQL injection group, XSS group), and label each group with the corresponding attack type label (such as "SQL injection - parameter variation class") to ensure that the candidate features in the same group all point to the same type of attack behavior, laying the classification foundation for subsequent feature selection.

[0093] In S13, for each set of candidate features of the same type of attack, the Longest Common Subsequence (LCS) algorithm is used to extract common core features. First, the number of candidate features in the group is determined. Assuming there are n candidate features, the first two are selected, and the longest consecutive common character sequence among these two features is found. This longest common character sequence is then compared with the third candidate feature, and the longest common character sequence among all three is found. This logic is iterated until all candidate features in the group are compared. The longest common character sequence among all candidate features is the common core feature of the attack group. The core logic of this algorithm is to compare candidate features one by one, prioritizing the retention of character sequences common to all candidates, while discarding unique variations or redundant parts of each candidate, ensuring that the finally extracted common core feature is the necessary feature for the implementation of this type of attack.

[0094] For the extracted common core features, they are broken down into indivisible smallest character sub-features according to the attack semantics (for example, "union select" is broken down into two sub-features: "union" and "select"). The information entropy weight of each sub-feature is calculated to quantify its representational value of the attack behavior. The specific calculation process of the information entropy weight includes: counting the number of times each sub-feature appears in all samples of the same type of attack in the group; dividing the number of occurrences of the sub-feature by the total number of samples of the same type of attack to obtain the probability of occurrence of the sub-feature in the same type of attack; calculating the information entropy value based on the probability of occurrence. The more stable the probability of occurrence of a sub-feature (i.e., most attack samples contain the sub-feature), the higher its information entropy value, indicating that the sub-feature is an indispensable core part of the attack behavior and has a stronger attack representation value; conversely, the more volatile the probability of occurrence of a sub-feature, the lower its information entropy value and the weaker its attack representation value.

[0095] In S14, based on the calculated information entropy weights of each sub-feature, a pre-set information entropy threshold is established (the threshold can be dynamically adjusted according to different attack types, for example, uniformly set to 1.0). The information entropy value of each sub-feature is compared with the information entropy threshold, and low-information-entropy-weight sub-features below the threshold are eliminated. These low-information-entropy-weight sub-features are mostly replaceable and deformable interference features (such as spaces and comment characters in SQL injection, which do not affect the attack even if deleted or replaced). Only high-information-entropy-weight sub-features above the information entropy weight threshold are retained. These high-information-entropy-weight sub-features are the smallest effective features that uniquely represent the essence of the attack and are indivisible (such as the core sub-feature "union" in SQL injection and the core sub-feature "script" tag in XSS). The smallest effective features selected under all attack types are summarized, and after removing duplicate features, a complete set of smallest effective attack features is constructed. Each feature in this set possesses the core attributes of being "indivisible and uniquely pointing to the essence of the attack."

[0096] In S2, refer to Figure 3 The flowchart illustrating the method for constructing the attack micro-feature cluster library shows the following steps: "Clustering the minimum effective feature set to generate attack micro-feature clusters and constructing the attack micro-feature cluster library":

[0097] S21: Perform two-layer semantic labeling on all minimum effective features within the minimum effective feature set. The first layer of semantic labeling is the attack category label, and the second layer of semantic labeling is the attack semantic sub-label.

[0098] S22: Group the smallest effective features of the same attack category label into candidate clusters;

[0099] S23: Based on the attack semantic sub-label, obtain the weighted similarity between the smallest effective features in each candidate cluster;

[0100] S24: Based on weighted similarity, cluster or split the candidate clusters to generate attack micro-feature clusters;

[0101] S25: Construct an attack micro-feature cluster library from the generated attack micro-feature clusters.

[0102] In S21, a two-layer semantic labeling operation is performed on all minimum effective features within the minimum effective feature set. In the first layer, attack category labels are uniformly labeled as core attack category labels such as "SQL injection," "cross-site scripting (XSS)," and "command injection" based on the attack type corresponding to the feature, ensuring consistent feature labeling for the same type of attack. In the second layer, attack semantic sub-labels are labeled with more granular semantic labels based on the attack implementation logic and function of the feature. For example, SQL injection features can be labeled with sub-labels such as "basic query keyword concatenation," "join query keyword concatenation," "Boolean blind injection feature trigger," and "time blind injection feature trigger," while XSS features can be labeled with sub-labels such as "script trigger," "DOM injection," and "tag bypass," achieving accurate semantic classification of features and providing semantic basis for subsequent clustering.

[0103] In S22, all minimum effective features labeled with double-layer semantic tags are traversed. Using the first-layer attack category label as the classification basis, all minimum effective features labeled with the same attack category label are grouped into the same candidate cluster, while features with different attack category labels belong to different candidate clusters. For example, all features labeled with the "SQL Injection" category label are grouped into the "SQL Injection Candidate Cluster," and all features labeled with the "XSS" category label are grouped into the "XSS Candidate Cluster." Through this classification method, feature isolation at the attack category level is achieved first, avoiding mixed clustering of different types of attack features and improving the accuracy of subsequent clustering.

[0104] In S23, for each candidate cluster, the focus is on the pairwise minimum effective features within the cluster, which means obtaining the character similarity, the matching degree of the attack semantic sub-label, and the association similarity of the information entropy weight among the minimum effective features within the candidate cluster; the character similarity, matching degree, and association similarity are weighted and summed to obtain the weighted similarity among the minimum effective features within the candidate cluster.

[0105] Specifically, for the character similarity of pairwise minimum effective features, a string comparison algorithm is used to quantify the matching degree of the character sequences of the two features. For the matching degree of attack semantic sub-labels, if the semantic sub-labels of the two features are completely identical, the matching degree is 100%; if they are partially identical, the matching degree is quantified according to the overlap ratio. Simultaneously, based on the information entropy weights of the two features, the association similarity is obtained by calculating F = 2 × min (W1, W2) / (W1 + W2), where W1 and W2 are two different minimum effective features within the candidate cluster, and F is the association similarity between W1 and W2 in the information entropy weight dimension. Preset weight coefficients are configured for the above three dimensions (e.g., character similarity 40%, matching degree 30%, association similarity 30%), and the values ​​of the three dimensions of each pair of features are weighted and summed to finally obtain the comprehensive weighted similarity between each pair of minimum effective features.

[0106] This scheme breaks through the limitations of traditional single-character-dimensional clustering, allowing the similarity determination between features to not only fit the character's appearance features, but also match the semantic essence of the attack and the core degree of the features, greatly improving the accuracy of similarity calculation and providing core data support for the accuracy of subsequent adaptive clustering.

[0107] In S24, based on the weighted similarity of the pairwise minimum effective features within a candidate cluster, a similarity threshold is set (which can be dynamically adjusted according to the attack category, such as a default of 85%), and adaptive clustering and sub-cluster splitting operations are performed on the candidate clusters. All features within a cluster are traversed, and the minimum effective features with weighted similarity higher than the threshold are aggregated into initial sub-clusters. The initial sub-clusters are then subjected to secondary verification. If the semantic sub-label matching degree of all minimum effective features within a sub-cluster meets the preset conditions, the sub-cluster is retained as the final attack micro-feature cluster. If there are semantic sub-label conflicts or features with low weighted similarity within a sub-cluster, independent sub-clusters are automatically split off until all features within all sub-clusters meet the similarity and semantic consistency requirements, ultimately generating multiple attack micro-feature clusters that accurately match the attack semantics.

[0108] In S25, all candidate attack categories are collected and adaptively clustered to generate attack micro-feature clusters. Each micro-feature cluster is assigned a unique identifier, labeled with the corresponding attack category label and core semantic sub-label. Simultaneously, all minimum effective features and feature matching weights within the cluster are recorded. A structured storage structure for the cluster library is designed, reserving storage fields and expansion interfaces for new attack micro-feature clusters to support incremental updates of subsequently added features. All completed attack micro-feature clusters are uniformly incorporated into the storage system, constructing a complete and dynamically expandable attack micro-feature cluster library.

[0109] The above steps construct an attack micro-feature cluster library with attack semantics as the core. It achieves accurate semantic classification of features through two-layer semantic labels, and effectively isolates features of different attack types by dividing candidate clusters based on attack categories. Combined with multi-dimensional weighted similarity calculation and adaptive clustering sub-cluster splitting, it enables the precise aggregation of scattered minimum effective features according to attack semantics. The final constructed attack micro-feature cluster library has both semantic relevance and structural rationality. Features within a cluster highly conform to the behavioral essence of the same type of attack, and features between clusters are free from semantic confusion. Moreover, the cluster library supports dynamic expansion, laying an accurate and efficient feature library foundation for subsequent real-time traffic micro-feature cluster matching.

[0110] In S3, lightweight acquisition modules deployed on web application gateways or traffic monitoring nodes capture all access traffic of web applications in real time (including normal access and potential attack access traffic). The acquisition process only captures core data that is strongly related to attack detection (URL, request method, POST body, parameter key-value pairs, etc.) and does not collect redundant traffic fields (such as irrelevant request headers and static resource loading traffic), reducing the computational power consumption of data transmission and subsequent processing. At the same time, the raw traffic is cached and standardized in real time (uniform character encoding is UTF-8) to avoid the distortion of subsequent feature extraction caused by the chaotic format of the raw traffic.

[0111] Based on the identified key payload regions, a combination of regular expression matching and semantic filtering is used to extract key features. First, a pre-defined regular expression (adapting to the minimum effective feature character pattern for attacks such as SQL injection and XSS) is used to initially filter out suspected attack features from the standardized core traffic data. Then, these suspected attack features are matched against attack semantic sub-labels. Suspected attack features that fail to match any of the attack semantic sub-labels are considered normal feature fragments without attack relevance and are discarded. Suspected attack features that successfully match the attack semantic sub-labels are key features of the access traffic that can characterize potential attack behavior and are extracted. This method ensures that the extracted key features are consistent with the feature dimensions of the attack micro-feature cluster library, adapting to subsequent matching requirements.

[0112] Reference Figure 4 The flowchart illustrating the statistical methods for key feature-related parameters shows that "statistical analysis of the current occurrence count, cumulative occurrence count, and combination methods of key features" includes the following steps:

[0113] S31: Count the number of times key features appear in the current access traffic;

[0114] S32: Count the cumulative number of times key features appear in all access traffic within a preset time window;

[0115] S33: Obtain the embedding position and splicing order of key features in the current access traffic, as well as the attack semantic logical relationship between each key feature;

[0116] S34: The way to combine key features by associating and integrating the embedding position, splicing order and attack semantic logic.

[0117] In S31-S34, each access traffic entry is used as an independent statistical unit. For all key features extracted from that traffic entry, a temporary frequency statistics dictionary is established. Each key feature is used as a dictionary key value. The character segments of the key payload region are traversed one by one. Each time a target key feature is matched, the corresponding dictionary value is incremented by 1. After the traversal is completed, the value corresponding to each key value in the dictionary is the current occurrence count of that key feature in the current access traffic. After the statistics are completed, the frequency statistics results of the single traffic entry are retained to provide a basis for subsequent multi-dimensional data integration.

[0118] Pre-configure statistical time windows (e.g., 5 seconds, 10 seconds, dynamically adjustable as needed) to adapt to web application access scenarios. For all access traffic within the preset time window, establish a global cumulative count table. Calculate the cumulative occurrence count of key features for each access traffic. If the same key feature appears in multiple traffic streams, sum the current occurrence counts in each traffic stream sequentially. The final value is the cumulative occurrence count of the key feature in all access traffic within the preset time window. When the time window slides and updates, traffic data exceeding the window range is simultaneously removed and the cumulative frequency is recalculated.

[0119] For the current access traffic (the access traffic at the latest moment), while extracting key features, the specific embedding position of each key feature in the key payload region is recorded, and the start and end position information of the feature is quantified by character offset; according to the actual order of appearance of the key features in the character stream of the current access traffic, a sequence encoding value is assigned to each feature (e.g., 1 for the first appearance and 2 for the second appearance) to represent the concatenation order of the features.

[0120] Iterates through all key features and performs two-layer judgment based on the inherent behavioral logic of web attacks. The first layer judges the matching of functional characters (such as whether the closing character ' / ' in SQL injection is adjacent to or matches the query keyword, and whether the angle brackets <> in XSS are adjacent to or match the script keyword). The second layer judges the order of core features (such as whether attack core features such as select and from, union and select are connected in semantic order). If both "matching of functional characters and orderly connection of core features" are satisfied, it is determined that there is an attack semantic logical relationship between the key features, and the relationship type is marked (such as SQL injection type, XSS type). If not satisfied, it is determined that there is no attack semantic logical relationship. The semantic judgment result is stored in the form of binary numerical value + type code (present = 1, absent = 0, and the relationship type is assigned a fixed code according to the attack category).

[0121] The acquired information, including embedding location, concatenation order, and attack semantic logical association, is structurally integrated and quantitatively represented. Specifically, the character offset of the embedding location is converted into a standardized value, the feature list of the concatenation order is converted into an ordered code, and the sequence code value of the concatenation order is scaled proportionally according to the number of features. The binary value of the attack semantic logical association plus the type code is directly used as the feature value. Taking a single access traffic instance as a unit, the standardized value of the location, the sequence code, and the overall semantic code corresponding to each key feature are bound together to form a unified structured data body. This data body fully covers the spatial distribution, sequence relationship, and semantic association of key features in the traffic, representing the combination method of key features in the current access traffic.

[0122] In S4, refer to Figure 5 The flowchart illustrating the cluster label acquisition method shows the following steps for obtaining cluster labels for attack micro-feature clusters:

[0123] S41: After clustering or splitting the candidate clusters to generate attack micro-feature clusters, the attack category labels contained in the clustered or split candidate clusters are decomposed into two-layer cluster labels.

[0124] S42: Use the double-layer cluster label as the exclusive cluster label for the corresponding attack micro-feature cluster;

[0125] S43: When a candidate cluster is directly used as an attack micro-feature cluster, the attack category label contained in the candidate cluster is used as the exclusive cluster label of the corresponding attack micro-feature cluster.

[0126] In S41-S43, if multiple candidate clusters are clustered to generate one attack micro-feature cluster, the same top-level attack domain label is added to the attack category label of these candidate clusters to form a two-layer cluster label of "top-level attack domain label - attack category label"; if one candidate cluster is split to generate two or more attack micro-feature clusters, a more refined bottom-level attack sub-semantic label is added to the attack category label of the candidate cluster to form a two-layer cluster label of "attack category label - bottom-level attack sub-semantic label". The core of this principle is to use attack category labels as the core layer, combined with another dimension label to form a two-layer cluster label. This two-layer cluster label is then uniformly assigned to all attack micro-feature clusters generated after clustering / splitting. For example, when two candidate clusters, "SQL injection" and "command injection", are clustered into one micro-feature cluster, a top-level attack domain label "code injection attack" is added uniformly. Then, it is decomposed into two-layer cluster labels, "code injection attack - SQL injection" and "code injection attack - command injection", which are then jointly assigned to the attack micro-feature clusters generated by this clustering. When the candidate cluster "SQL injection" is split into two or more micro-feature clusters, it is split into two-layer cluster labels, "SQL injection - keyword concatenation" and "SQL injection - blind injection trigger", which are then assigned to each attack micro-feature cluster generated after splitting. This achieves precise matching between cluster labels and the attack attribute level of the clusters. If a candidate cluster does not undergo any clustering or sub-cluster splitting operations and is directly treated as a single attack micro-feature cluster, then there is no need for label splitting. The original attack category label of the candidate cluster is directly assigned to its corresponding attack micro-feature cluster as the unique cluster label of the cluster. Throughout the process, it is ensured that each attack micro-feature cluster has a unique corresponding cluster label, and the label assignment rule is consistent with the clustering processing logic of the candidate cluster. In addition, based on the inherent nature of attack features, the sub-clusters obtained after splitting are unlikely to undergo secondary clustering with other clusters. Therefore, the above-mentioned attack category label splitting method can fully meet the needs of practical applications.

[0127] After refining the cluster labels of the attack micro-feature cluster library, key features are used for retrieval. The target minimum effective feature with the highest similarity value to the key feature is selected from the attack micro-feature cluster library. The matching result is composed of the key feature, the target minimum effective feature, the similarity value, the attack semantic sub-label of the target minimum effective feature, and the exclusive cluster label of the attack micro-feature cluster to which the target minimum effective feature belongs. The similarity value is the character similarity between the key feature and the minimum effective feature. The minimum effective feature with the highest similarity value to the key feature is taken as the target minimum effective feature, and its related three-layer labels are added to the matching result. This serves as supplementary semantic information for the detection model, while the similarity value corresponds to the credibility of the supplementary semantic information.

[0128] In S5, refer to Figure 6The flowchart illustrating the structured numerical matrix construction method shows the steps involved in "constructing a structured numerical matrix based on matching results, the current occurrence count, cumulative occurrence count, and combination method of key features":

[0129] S51: Convert non-numeric fields in the matching results and combination methods into fixed encoded values;

[0130] S52: Normalize the numerical fields in the current occurrence count, cumulative occurrence count, matching results, and combination methods into standardized values;

[0131] S53: Construct a structured numerical matrix based on fixed encoded values ​​and standardized numerical values.

[0132] A structured numerical matrix is ​​constructed using the unique ID of the key feature as the row dimension and the standardized multi-dimensional data as the column dimension.

[0133] In S6, the detection model includes an input layer, a feature extraction layer, a classification decision layer, and an output layer. The input layer is used to convert the structured numerical matrix into tensor data; the feature extraction layer is used to mine multi-dimensional feature association patterns in the tensor data to generate high-dimensional feature vectors; the classification decision layer is used to obtain the probability distribution of the high-dimensional feature vectors corresponding to two categories of labels: malicious attack and normal traffic, and outputs the malicious attack judgment result based on the probability distribution; the output layer is used to convert the malicious attack judgment result into a standardized format and output the detection result.

[0134] The core objective of the input layer is to convert the structured numerical matrix into a model-recognizable tensor and perform data validation. First, it receives the constructed structured numerical matrix and converts it from a two-dimensional array format into tensor data compatible with deep learning frameworks. The tensor dimensions are fixed at [batch size, number of features, feature dimension], where the batch size is set to 1 for real-time traffic detection to ensure real-time processing of individual traffic. Then, it performs data compliance validation, including dimension validation and numerical range validation. Dimension validation confirms that the feature dimension matches the preset number of neurons in the input layer; if inconsistent, it either fills in the gaps with zeros or filters core dimensions based on feature importance. Numerical range validation ensures that standardized values ​​are in the 0-1 range and fixed encoded values ​​are within the preset encoding range. After removing outliers, the compliant tensor data is passed to the feature extraction layer. The entire process takes less than 1ms, meeting the low-latency requirements of real-time web traffic detection.

[0135] The core of the feature extraction layer is to uncover the semantic association patterns between discrete numerical features. It employs a two-layer fully connected neural network as its core structure. The first fully connected layer has twice the number of neurons as the input feature dimension. A linear transformation is performed on the input tensor using a weight matrix, mapping discrete features such as similarity, current occurrence count, cumulative occurrence count, and combination method in the structured numerical matrix into linear combination features in a high-dimensional space. The second fully connected layer has half the number of neurons and introduces a non-linear transformation using the ReLU activation function to capture non-linear associations between features, such as the mapping relationship between the combination of "high similarity + high occurrence count + semantic association with attack" and malicious attacks. A Dropout layer is connected after the two fully connected layers, setting a 20% random deactivation probability for neurons to prevent the model from overfitting to the attack features on the training set and improving its generalization ability to unknown deformed attack features. Finally, the feature vector after non-linear transformation and overfit suppression is compressed to produce a high-dimensional feature vector with a dimension half that of the original input. This vector contains the core information of "feature association patterns + semantic attributes of attack" and is directly passed to the classification decision layer.

[0136] The core of the classification decision layer is to perform probabilistic determination of malicious attacks based on high-dimensional feature vectors. It connects to a Softmax classifier, using the Softmax function to calculate the probability distribution of the high-dimensional feature vectors corresponding to the two labels "malicious attack" and "normal traffic," outputting the probability value for each label (range 0-1). Simultaneously, an L2 regularization mechanism is introduced to suppress overfitting of the weight matrix, ensuring the stability of probability calculation and avoiding judgment bias caused by feature noise. A preset confidence threshold for malicious attack determination is set, defaulting to 85%, which can be dynamically adjusted according to actual business scenarios. The "malicious attack" probability value output by Softmax is compared with the threshold. If the probability reaches or exceeds the threshold, the current traffic is determined to be a malicious attack, and the probability value is recorded as the determination confidence level; if the probability is below the threshold, it is determined to be normal traffic. After the determination, the core information of "determination result (malicious / normal) + confidence level value" is passed to the output layer. Through probabilistic decision-making rather than hard judgment, the precision and recall of detection are balanced, effectively reducing the risk of false positives and false negatives.

[0137] The core of the output layer is to transform the judgment results into a standardized format and trigger subsequent actions. It receives the judgment results and confidence scores from the classification decision layer, converting them into a standardized key-value pair format containing three core types of information: "detection result label (malicious attacks are labeled 1, normal traffic is labeled 0)," "confidence score (a value between 0 and 1)," and "core judgment features (such as the attack micro-feature cluster ID with the highest matching degree, and the ID of frequently occurring key features)." Then, based on the detection results, corresponding actions are triggered. If the label is 1 (i.e., judged as a malicious attack), a blocking operation trigger signal is output. The signal uses a standardized interface format and can directly interface with the blocking module of a Web application firewall. If the label is 0 (i.e., judged as normal traffic), only a detection result log is output, containing key information such as traffic ID, judgment time, and confidence score. All output information is in JSON format to ensure compatibility with subsequent blocking and log analysis modules, while retaining complete judgment tracing information to provide data support for subsequent model performance optimization.

[0138] In S7, the difference between the output of the detection model and the feature matching of the attack micro-feature cluster library determines whether the currently detected malicious attack is a new type of unknown attack. That is, after the detection model outputs the judgment result of malicious attack, the matching result of the key feature of the traffic is extracted. If the character similarity between the key feature and all the smallest effective features in the attack micro-feature cluster library is lower than the preset low threshold (such as 50%), and the confidence of the detection model is ≥70% (indicating that the traffic has malicious features but no matching cluster), then the attack is marked as a new type of unknown attack. At the same time, the high-dimensional feature vector output by the feature extraction layer is compared with the core feature vector distribution of the existing attack micro-feature clusters in the cluster library. If the vector falls outside the feature distribution range of all known clusters, it is further verified that it is a new type of unknown attack. The dual judgment rule avoids misjudging deformed known attacks as unknown attacks and ensures the accuracy of unknown attack identification.

[0139] After confirming a new type of unknown attack, an incremental clustering algorithm is used to update its features in the cluster database. First, the minimum effective features of the new unknown attack are extracted, retaining its standardized character features, attack semantic association patterns, and other core attributes. This avoids re-traversing the entire historical feature set; incremental calculations are performed only on existing attack micro-feature clusters in the cluster database. The similarity between the new feature and each known cluster is calculated. If the similarity is ≥60%, it is assigned to the corresponding cluster, and the cluster's feature distribution range is updated. If all similarities are <60%, a new attack micro-feature cluster is generated, assigned a unique cluster ID and a three-layer semantic label (top-level attack domain label categorized based on attack principles, attack category label divided according to core harmful objects, and bottom-level sub-labels annotating specific attack methods). Subsequently, the new cluster or the updated cluster is synchronized to the attack micro-feature cluster database, and the three-layer association index table of the cluster database is updated. Clustering calculations are performed only on newly added features throughout the process, avoiding the high computational cost of full clustering, adapting to the need for real-time cluster database updates, and ensuring that subsequent detection can quickly identify this type of new unknown attack.

[0140] Reference Figure 7 This disclosure provides an AI-based web application attack detection and defense system, including:

[0141] Extraction module 101 is used to acquire historical Web attack traffic data and extract the minimum effective feature set of attack behavior from the historical Web attack traffic data;

[0142] Clustering module 102 is used to cluster the minimum effective feature set, generate attack micro-feature clusters and build an attack micro-feature cluster library;

[0143] The statistics module 103 is used to collect access traffic of Web applications, extract key features of access traffic, and count the current occurrence, cumulative occurrence, and combination of key features.

[0144] Matching module 104 is used to perform attack micro-feature cluster matching on key features based on the attack micro-feature cluster library to obtain matching results;

[0145] Module 105 is used to construct a structured numerical matrix based on the matching results, the current occurrence count, cumulative occurrence count, and combination method of key features, and input the structured numerical matrix into the detection model;

[0146] The execution module 106 is used to perform blocking operations when the detection model outputs the detection result of malicious attack;

[0147] The update module 107 is used to update the features of the new unknown attack to the attack micro-feature cluster library through an incremental clustering algorithm when the malicious attack is identified as a new type of unknown attack.

[0148] The various variations and specific examples of the AI-based Web application attack detection and defense method provided above are also applicable to the AI-based Web application attack detection and defense system provided in this disclosure. Through the foregoing detailed description of the AI-based Web application attack detection and defense method, those skilled in the art can clearly understand the implementation method of the AI-based Web application attack detection and defense system. For the sake of brevity, it will not be described in detail here.

[0149] A computer device according to embodiments of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0150] The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of this disclosure, the processor is used to execute computer-readable instructions stored in the memory, causing the computer device to perform all or part of the steps of the AI-based Web application attack detection and defense methods described in the foregoing embodiments of this disclosure.

[0151] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0152] like Figure 8 This is a schematic diagram of a computer device provided for an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the computer device in the embodiments of the present disclosure. Figure 8 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0153] like Figure 8As shown, a computer device may include a processor (such as a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from storage devices into random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0154] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow the computer device to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 8 A computer apparatus with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0155] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the AI-based Web application attack detection and defense method of embodiments of this disclosure are performed.

[0156] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0157] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the AI-based web application attack detection and defense methods described in the foregoing embodiments of the present disclosure are performed.

[0158] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0159] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0160] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0161] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0162] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0163] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0164] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0165] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0166] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for detecting and defending against web application attacks based on AI, characterized in that, include: Obtain historical web attack traffic data and extract the minimum effective set of attack behavior features from the historical web attack traffic data; Cluster the minimum effective feature set to generate attack micro-feature clusters and construct an attack micro-feature cluster library; Collect access traffic of web applications, extract key features of the access traffic, and count the current occurrence count, cumulative occurrence count, and combination methods of the key features; Based on the attack micro-feature cluster library, the key features are matched using attack micro-feature cluster matching to obtain matching results; Based on the matching results, the current occurrence count, cumulative occurrence count, and combination method of the key features, a structured numerical matrix is ​​constructed, and the structured numerical matrix is ​​input into the detection model; When the detection model outputs a result indicating a malicious attack, a blocking operation is performed; When the malicious attack is identified as a novel unknown attack, the features of the novel unknown attack are updated to the attack micro-feature cluster library through an incremental clustering algorithm.

2. The AI-based Web application attack detection and defense method according to claim 1, characterized in that, The process of acquiring historical web attack traffic data and extracting the minimum effective set of attack behavior features from the historical web attack traffic data includes: The acquired historical web attack traffic data is preprocessed to obtain structured core traffic data; Extract all character fragments of key payload regions from the core traffic data of similar attacks as candidate features, and group and label the candidate features of the same type according to the attack type. Extract common core features from the candidate features of each group of attacks of the same type, split the common core features into sub-features of the smallest character, and obtain the information entropy weight of the sub-features. Based on the information entropy weight, the minimum effective feature is selected from the sub-features, and the selected minimum effective features constitute the minimum effective feature set.

3. The AI-based Web application attack detection and defense method according to claim 1, characterized in that, The step of clustering the minimum effective feature set to generate attack micro-feature clusters and constructing an attack micro-feature cluster library includes: A two-layer semantic labeling is performed on all minimum effective features within the minimum effective feature set. The first layer of semantic labeling is the attack category label, and the second layer of semantic labeling is the attack semantic sub-label. The smallest effective features of the same attack category label are grouped into candidate clusters; Based on the attack semantic sub-label, obtain the weighted similarity between the minimum effective features within the candidate cluster; Based on the weighted similarity, the candidate clusters are clustered or split to generate attack micro-feature clusters; The generated attack micro-feature clusters are used to construct an attack micro-feature cluster library.

4. The AI-based Web application attack detection and defense method according to claim 3, characterized in that, The step of obtaining the weighted similarity between the minimum effective features within the candidate cluster based on the attack semantic sub-label includes: Obtain the character similarity, the matching degree of attack semantic sub-labels, and the association similarity of information entropy weights among the minimum effective features within the candidate cluster; The weighted sum of the character similarity, the matching degree, and the association similarity is obtained to obtain the weighted similarity between the smallest effective features within the candidate cluster.

5. The AI-based Web application attack detection and defense method according to claim 1, characterized in that, The statistics on the current occurrence count, cumulative occurrence count, and combination methods of the key features include: Count the number of times the key feature appears in the current access traffic; The cumulative number of times the key feature appears in all access traffic within a preset time window is counted. Obtain the embedding position and splicing order of the key features in the current access traffic, as well as the attack semantic logical relationship between each key feature; The embedding location, splicing order, and attack semantic logic are associated and integrated to form the combination method of the key features.

6. The AI-based Web application attack detection and defense method according to claim 3, characterized in that, Also includes: After clustering or splitting the candidate clusters to generate attack micro-feature clusters, the attack category labels contained in the clustered or split candidate clusters are decomposed into two-layer cluster labels. The dual-layer cluster label is used as the exclusive cluster label for the corresponding attack micro-feature cluster; When the candidate cluster is directly used as the attack micro-feature cluster, the attack category label contained in the candidate cluster is used as the exclusive cluster label of the corresponding attack micro-feature cluster.

7. The AI-based Web application attack detection and defense method according to claim 6, characterized in that, The process of matching attack micro-feature clusters on the key features based on the attack micro-feature cluster library to obtain matching results includes: The smallest effective target feature with the highest similarity value to the key feature is selected from the attack micro-feature cluster library; The matching result is composed of the key features, the target minimum effective features, the similarity value, the attack semantic sub-labels of the target minimum effective features, and the exclusive cluster labels of the attack micro-feature clusters to which the target minimum effective features belong.

8. The AI-based Web application attack detection and defense method according to claim 1, characterized in that, The construction of a structured numerical matrix based on the matching results, the current occurrence count, cumulative occurrence count, and combination method of the key features includes: Convert the non-numeric fields in the matching results and the combination methods into fixed encoded values; The numerical fields in the current occurrence count, the cumulative occurrence count, the matching result, and the combination method are uniformly normalized into standardized values; A structured numerical matrix is ​​constructed based on the fixed encoded value and the standardized numerical value.

9. The AI-based Web application attack detection and defense method according to claim 1, characterized in that, The detection model includes an input layer, a feature extraction layer, a classification decision layer, and an output layer; The input layer is used to convert the structured numerical matrix into tensor data; The feature extraction layer is used to mine multi-dimensional feature association patterns in the tensor data and generate high-dimensional feature vectors. The classification decision layer is used to obtain the probability distribution of two types of labels, malicious attack and normal traffic, corresponding to the high-dimensional feature vector, and output the malicious attack judgment result based on the probability distribution. The output layer is used to convert the malicious attack judgment result into a standardized format and output the detection result.

10. An AI-based web application attack detection and defense system, characterized in that, The system includes: The extraction module is used to acquire historical web attack traffic data and extract the minimum effective feature set of attack behavior from the historical web attack traffic data. The clustering module is used to cluster the minimum effective feature set, generate attack micro-feature clusters, and construct an attack micro-feature cluster library; The statistics module is used to collect access traffic of the web application, extract key features of the access traffic, and count the current occurrence count, cumulative occurrence count, and combination method of the key features; The matching module is used to perform attack micro-feature cluster matching on the key features based on the attack micro-feature cluster library to obtain the matching result; The construction module is used to construct a structured numerical matrix based on the matching results, the current occurrence count, cumulative occurrence count, and combination method of the key features, and input the structured numerical matrix into the detection model; The execution module is used to perform a blocking operation when the detection model outputs a detection result of a malicious attack; The update module is used to update the features of the new unknown attack to the attack micro-feature cluster library by using an incremental clustering algorithm when the malicious attack is identified as a new unknown attack.

Images