A web anomaly detection system and method

Abstract

The application discloses a WEB exception detection system and method, wherein the method comprises the following steps: classifying historical WEB request records of different users into different API interfaces, and encoding different API interface values into API ID sequences; creating a plurality of exception detection models for detecting abnormal WEB requests based on the API ID sequences; using each exception detection model to respectively detect whether a target WEB request is an abnormal request; an exception detection model with a detection result of an abnormal request generates an alarm; and whether the risk level of the IP address of the WEB request is raised is judged by counting whether the number of the exception detection models that generate the alarm meets a preset threshold value. The WEB request is converted into data that can be directly used by the exception detection model, and the effects of a plurality of exception detection models are comprehensively used, so that the detection accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a web anomaly detection method and a web anomaly detection device. Background Technology

[0002] For enterprises, various web services are frequently exposed to the outside world, and vulnerabilities in these services are often potential entry points for security attacks. Traditional rule-based anomaly detection systems are often used to detect attacks targeting web services, but they are sometimes overly reliant on business systems and have poor portability. In recent years, deep learning-based web anomaly detection systems have become popular. Their detection process often requires labeled data; however, labeling logs requires a significant amount of manpower, making it difficult to ensure that the training set for supervised methods is adequately obtained. Summary of the Invention

[0003] In view of the above problems, the present invention is proposed to provide a web anomaly detection system and method that overcomes or at least partially solves the above problems.

[0004] This application discloses a web anomaly detection method, which includes:

[0005] The historical web request records of different users are categorized into different API interfaces, and the numerical values ​​of different API interfaces are encoded into API ID sequences;

[0006] Create multiple anomaly detection models based on the API ID sequence to detect abnormal web requests;

[0007] Each of the aforementioned anomaly detection models is used to detect whether the target web request is an abnormal request.

[0008] An anomaly detection model generates an alert if the detection result indicates an abnormal request.

[0009] By statistically analyzing whether the number of alarms issued by the anomaly detection model meets a preset threshold, it is determined whether to increase the risk level of the IP address of the web request.

[0010] In one embodiment, classifying historical web requests from different users into different API interfaces includes: analyzing and identifying the API corresponding to the web request by employing parameter elimination.

[0011] In one embodiment, the URLs obtained after parameter elimination are categorized, with each URL corresponding to an API ID; the parameter elimination includes:

[0012] Remove query parameters starting with "?" from the URL to identify whether they are similar API interfaces; or

[0013] Remove long strings or numbers exceeding a specified threshold from a URL's path at a certain level and replace them with appropriate placeholders to identify whether they belong to the same type of API; or

[0014] Remove static filenames from the last-level path of the URL, retaining only the file extension to identify whether they belong to the same type of API; or

[0015] Each distinct path in the URL is treated as a path parameter and replaced with a placeholder to identify whether they are API interfaces of the same type.

[0016] In one embodiment, creating multiple anomaly detection models for detecting abnormal web requests based on the API ID sequence includes:

[0017] The API ID sequence is divided into multiple substrings, each substring corresponding to a web request;

[0018] An AC automaton is built based on multiple substrings to serve as an anomaly detection model for identifying abnormal web requests;

[0019] In one embodiment, creating multiple anomaly detection models for detecting abnormal web requests based on the API ID sequence includes:

[0020] Attach a pointer to a node on the AC automaton to each user's session;

[0021] If the current node of a web request moves to the root node, the web request does not match any of the substrings and is identified as an abnormal web request.

[0022] In one embodiment, creating multiple anomaly detection models for detecting abnormal web requests based on the API ID sequence includes:

[0023] The anomaly detection model is a Transformer model. The API ID sequence is input into the Transformer model to obtain the reconstructed API ID. The abnormal web request is identified by the deviation between the API ID sequence before and after reconstruction.

[0024] In one embodiment, creating multiple anomaly detection models for detecting abnormal web requests based on the API ID sequence includes:

[0025] The anomaly detection model is an LSTM Predictor model. By inputting the API ID sequence given by the LSTM Predictor model, it outputs the next API ID. If the probability of the next API ID meets a predetermined threshold, the web request is normal; otherwise, the web request is an abnormal web request.

[0026] This application also discloses a web anomaly detection system, including:

[0027] The data preprocessing module is used to classify the historical web request records of different users into different API interfaces and encode the different API interface values ​​into API ID sequences;

[0028] An anomaly detection model creation module is used to create multiple anomaly detection models for detecting abnormal web requests based on the API ID sequence;

[0029] The abnormal request alarm module generates alarms when the detection result is an abnormal request, based on the anomaly detection model.

[0030] The alarm processing module determines whether to increase the risk level of the IP address of the web request by statistically analyzing whether the number of anomaly detection models of the issued alarms meets a preset threshold.

[0031] In one embodiment, it includes: an API identification module, which analyzes and identifies the API corresponding to a web request by employing parameter elimination.

[0032] In one embodiment, the API identification module further includes: classifying the URLs obtained after parameter elimination, with each URL corresponding to an API ID; the parameter elimination includes:

[0033] Remove query parameters starting with "?" from the URL to identify whether they are similar API interfaces; or

[0034] Remove long strings or numbers exceeding a specified threshold from a URL's path at a certain level and replace them with appropriate placeholders to identify whether they belong to the same type of API; or

[0035] Remove static filenames from the last-level path of the URL, retaining only the file extension to identify whether they belong to the same type of API; or

[0036] Each distinct path in the URL is treated as a path parameter and replaced with a placeholder to identify whether they are API interfaces of the same type.

[0037] The beneficial effects of this invention are as follows: The web anomaly detection system and method of this application employ a model capable of actively learning from logs containing only normal behavior and extracting sequence information without requiring specific API documentation. This application uses a data preprocessing module to convert network traffic into an APIID sequence that can be used as input by the anomaly detection model. This application solves the problem of unsatisfactory detection results when using a single model, achieving better detection results by combining the advantages of three different anomaly detection models. Attached Figure Description

[0038] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0039] Figure 1 This is a flowchart of a web anomaly detection method provided in an embodiment of this disclosure;

[0040] Figure 2 This is a flowchart of a web anomaly detection system provided in an embodiment of this disclosure;

[0041] Figure 3 The sequence reconstruction process used to implement the Reconstruction with Transformer model in the embodiments of this disclosure.

[0042] Figure 4 This is a bar chart showing the number of alarms additionally discovered in one embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] In a web attack scenario, an ideal anomaly detection system needs to have the following four characteristics:

[0045] First, it requires no labeled data (or only a few labels). Second, it is not rule-based, as rules need constant updates to cope with evolving attack methods. Third, it does not rely on API documentation; it can self-discover web-accessible APIs through algorithms, thus possessing portability. Fourth, it should have fast processing speed and consume appropriate hardware resources to ensure no impact on normal business operations. The web anomaly detection method and system of this application meet the above four characteristics. To achieve an ideal web anomaly detection method and system with the above four characteristics, the system or device is called SeqTrident. This application also discloses the method and effects of SeqTrident in real-world scenarios.

[0046] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0047] Reference Figure 1 The diagram shows a flowchart of the web anomaly detection method according to Embodiment 1 of the present invention. Step S101: Classify the historical web request records of different users into different API interfaces, and encode the different API interface values ​​into API ID sequences.

[0048] In this embodiment of the invention, step 101 corresponds to the Data preprocessor in the workflow diagram, which converts web requests into numerical types such as API ID sequences. These API ID sequences are used for identification by the subsequent anomaly detection model. In this application, web requests from different users are first separated using cookies. Specifically, when a user accesses the website, the Data preprocessor assigns a unique number to the user, recorded as a session ID, and stores it in a cookie. Based on the session ID, access IP address, and other data, web requests from different users are processed separately.

[0049] Secondly, the corresponding API interface needs to be identified. The anomaly detection module processes numeric types, while network requests are in the format of HTTP messages, making direct detection inconvenient. Therefore, after separating web requests from different users, the Data preprocessor needs to encode the web requests into numeric values.

[0050] Generally, web applications place different business logic under different APIs. Even if the parameters are different, the server performs similar actions when a user accesses the same API. Therefore, the data preprocessor uses APIs for classification, treating user access to the same API as the same behavior, and removing other parameters in the message and ignoring them. We call the number of the API interface corresponding to a network request the API ID.

[0051] To avoid manually writing rules based on API documentation and to increase the portability of SeqTrident, a parameter elimination method is used to analyze and identify the API corresponding to a web request, obtaining the API ID for that API. Specifically, the URLs obtained after parameter elimination are categorized, with each URL corresponding to an API ID. For example, general rules are used to analyze and identify the API corresponding to a network request. These rules are as follows:

[0052] Query parameter elimination is a rule that directly truncates most of the query parameters starting with a "?" in the URL. Specifically, in real-world business systems, some query parameters determine the corresponding API, such as the `method` and `path` fields. In such cases, we will retain the relevant query parameters.

[0053] The rule for eliminating numbers or long strings considers a path at a certain level in the URL. If it is a number, a hexadecimal number, or a long string whose length exceeds a specified threshold (the string length threshold is set to 32 in this application), we treat it as a path parameter and replace it with the corresponding placeholder.

[0054] For static filename deletion, this rule considers the last level of the URL path. If the filename ends with an extension of a common static file type (such as JS, image, video, document), we treat the filename as a parameter and remove it, keeping only the extension.

[0055] Based on statistical elimination, this rule performs statistical analysis on the URLs eliminated by the above rules. If a set of URLs is found to be larger than a threshold (the threshold set in the experimental section of this paper is 30), and each URL has one and only one unique path while all other parts are identical, then the unique path in these URLs is treated as a path parameter and replaced with a placeholder.

[0056] Table 1 provides some examples of API identification. For each URL obtained after parameter elimination, we treat it as a category and assign it an API ID.

[0057] Table 1: Examples of API Recognition

[0058]

[0059] Step S102: Create multiple anomaly detection models based on the API ID sequence to detect abnormal web requests, and use each anomaly detection model to detect whether the target web request is an abnormal request.

[0060] In this application, step S101 converts multiple web requests into a sequence of API IDs issued by a single user. The next question is how to detect this API ID sequence. This application employs three different anomaly detection models (detectors) based on API ID sequences.

[0061] The AC Automaton with Possibility Partition model is presented as the first anomaly detection model in this application. Specifically, the API ID sequence is divided into multiple substrings, each substring corresponding to a web request. An AC automaton is built based on these substrings to serve as an anomaly detection model for identifying abnormal web requests. A pointer to a node on the AC automaton is appended to each user's session. If the current node of a web request moves to the root node, the web request does not match any of the substrings and is identified as an abnormal web request.

[0062] In detail, during the training phase, in order to learn what normal web request behaviors exist, each API ID sequence in normal traffic is divided into multiple segments according to a preset probability, with each segment corresponding to a web request behavior.

[0063] Specifically, let r[i] represent the i-th arriving web request. For consecutive web requests a and b, we calculate the probability across the entire API ID sequence dataset:

[0064] P(r[i]=a|r[i+1]=b)=P(ab) / P(r[i+1]=b)

[0065] P(r[i+1]=b|r[i]=a)=P(ab) / P(r[i]=a)

[0066] At the same time, we count the number of all possible predecessors of web request b, pre(b), and the number of all successors of web request a, next(a).

[0067] If the following conditions are met:

[0068] P(r[i]=a|r[i+1]=b)>1 / pre(b) or P(r[i+1]=b|r[i]=a)>1 / next(a)

[0069] If the probability of web request b following web request a is greater than average, or the probability of request a following request b is greater than average, then we consider web requests a and b to occur consecutively with a high probability and can group them into the same request segment. In this way, a continuous sequence of API IDs can be divided into multiple substrings, with each substring containing requests corresponding to a specific action. These substrings are then used to build an AC automaton to identify web request anomalies.

[0070] During the detection phase, a pointer is appended to each user's session by specifying a cookie. This pointer points to a node on the Aho-Corasick automaton. As the data preprocessor continuously identifies new API IDs, the user's corresponding node will continuously shift. According to the properties of the Aho-Corasick automaton, if the user's current node shifts to the root node, it means that the user's behavior does not conform to any substring and can be identified as a web request exception.

[0071] The second anomaly detection model used in this application is the Transformer model. The API ID sequence is input into the Transformer model to obtain the reconstructed API ID. Abnormal web requests are identified by the deviation between the API ID sequences before and after reconstruction. This application's Transformer model is based on the ideas of existing Transformer models and is named "Reconstruction with Transformer".

[0072] The existing Transformer model's core idea is to first encode the original sequence into latent variables, and then attempt to decode them back into the original sequence. Through extensive training, the goal is to extract as many rich features as possible from the latent variables to reconstruct the sequence. Sequence reconstruction is a method for anomaly detection without requiring abnormal data. Abnormal data in a sequence is easier to identify because it is difficult to reconstruct.

[0073] In the original Transformer mechanism, the mask operation prevents the prediction of a value at a certain position from utilizing information from all subsequent positions. By masking the part that needs to be reconstructed and using other parts to predict it, the Transformer can be used for sequence reconstruction. Anomaly detection can be achieved by comparing the deviations between the reconstructed and reconstructed sequences.

[0074] like Figure 3As shown, this application applies the Transformer model to anomaly detection in API ID sequences. A sliding window is used to traverse the API ID sequence with a fixed step size. For each API ID, its value is masked, and the API ID sequence is input into the Transformer model to obtain the reconstructed API ID. The reconstruction is then compared with the original API ID to determine if the reconstruction was successful.

[0075] The third anomaly detection model used in this application is the LSTM Predictor model. By inputting the API ID sequence given by the LSTM Predictor model, the next API ID is output. If the probability of the next API ID meets a predetermined threshold, the web request is normal; otherwise, the web request is an abnormal web request.

[0076] This model is characterized by using past outputs as current inputs, thus incorporating past results into its predictions. The LSTM Predictor model is a recurrent neural network with the ability to remember long-term dependencies. This model can be applied to anomaly detection in program logs. The LSTM Predictor model in this application has been trained on normal API ID sequences. Given a sequence of API IDs, it outputs the distribution of the next API ID's value. During the detection phase, the LSTM Predictor uses a sliding window to traverse the API ID sequence, using the current API ID sequence to predict the actual arriving API ID sequence. If the actual arriving web request belongs to one of the highest predicted probabilities, the request is considered normal; otherwise, the web request is considered abnormal, and an alert is issued.

[0077] Step S103: The anomaly detection model generates an alarm for an abnormal request.

[0078] Step S104: By statistically analyzing whether the number of alarms issued by the anomaly detection models meets a preset threshold, it is determined whether to increase the risk level of the IP address of the web request. By statistically analyzing the alarms generated by the three anomaly detection models, an intuitive threat classification is provided to operations and maintenance personnel. For alarms based on API ID sequences, if an attacker violates the behavioral logic of a normal user, each malicious web request they issue will trigger a warning from the model. To avoid alarm fatigue and filter out truly important alarms, Alarm Merger aggregates alarms using the IP address of network requests.

[0079] When a web request is processed by three models, the Alarm Merger records the number of models that triggered the alert. If multiple network requests from a certain IP address generate model alerts, and the sum of the number of alert models triggered by these network requests reaches a certain threshold, the risk level of that IP address will be increased.

[0080] Alarm Merger works in conjunction with the Security Operations Center (SOC) of the deployment environment to output alarm information to the SOC, so that operations and maintenance personnel can take action based on information such as alarm level and model prediction results.

[0081] This application also discloses a web anomaly detection system, including: a data preprocessing module, an anomaly detection model creation module, an anomaly request alarm module, and an alarm comprehensive processing module. See [link to relevant documentation]. Figure 2 The system can be briefly described as follows: When a network request (web request) is received, the Data preprocessor converts it into a model-readable input API ID. This API ID is then fed into three different sequence-based anomaly detection models (…). Figure 2 The AC (Automaton with Possibility Partition), Reconstruction with Transformer, and LSTM Predictor are used for detection, and their training does not depend on malicious data. If a model determines that a web request is abnormal, it generates an alert. Figure 2 Alarm in the context. Alarm processing module (corresponding to...) Figure 2 The Alarm Merger in the system takes into account the alarms generated by the three models and assigns different security levels to the alarms.

[0082] The data preprocessing module is used to classify the historical web request records of different users into different API interfaces and encode the different API interface values ​​into API ID sequences;

[0083] An anomaly detection model creation module is used to create multiple anomaly detection models for detecting abnormal web requests based on the API ID sequence;

[0084] The abnormal request alarm module generates alarms when the detection result is an abnormal request, based on the anomaly detection model.

[0085] The alarm processing module determines whether to increase the risk level of the IP address of the web request by statistically analyzing whether the number of anomaly detection models of the issued alarms meets a preset threshold.

[0086] To evaluate the performance of the three anomaly detection models in the web anomaly detection system (SeqTrident system), and the effect of their combined use in detection, we deployed the system in a company and participated in its large-scale real-world attack and defense training.

[0087] In a specific use case, the web anomaly detection system provided protection for 58 different business websites under two primary domains, involving more than 10^4 APIs. The website traffic fluctuated at different times of the week, with a maximum of over ten million requests received in a single day. The attack and defense training lasted 14 days, during which all of the company's websites were exposed to the real external world, and numerous hackers were invited to conduct penetration tests using advanced attack methods. In addition to deploying the web anomaly detection system described in this application, the company also deployed other algorithms from more than ten different security vendors, each independently detecting anomalies in received traffic.

[0088] During the training, alarms generated by the SeqTrident system were monitored in real time, manually verified, and compared with alarms generated by other security algorithms.

[0089] Before the attack and defense training began, we collected the company's access traffic during normal business hours on multiple occasions, accumulating two weeks' worth of training data and more than 10^8 request logs. Using this data, we conducted offline training on three sequence anomaly detection methods.

[0090] Based on the parameter preprocessing method mentioned in the data preprocessing module, we successfully categorized 10^8 request logs into 12450 different API interfaces. The different API interface values ​​are encoded into API ID sequences, which facilitates model training.

[0091] One-eighth of the entire API ID sequence data is selected as the validation set. Since all business traffic is collected outside of training periods, all logs can be considered normal behavior. We use the aforementioned API ID sequence data as training data to train an anomaly detection model, and then use the anomaly detection model to test the validation set to check for false positives. Based on the three anomaly detection models mentioned in this application, the following points should be noted during training:

[0092] 1) The first anomaly detection model is a model based on probability and AC automata. Since it is not a deep learning method, it only needs to traverse the training dataset multiple times to complete the model training.

[0093] 2) For the second and third anomaly detection models, due to the large size of the API ID sequence data, it is difficult to read it all into memory. Therefore, the data is read from the hard drive in batches. The Adam optimizer is used, and the model is trained over multiple epochs.

[0094] To improve detection accuracy, this application proposes a method for optimizing anomaly detection models, which includes: tuning detection accuracy and runtime load. The specific method for tuning detection accuracy is to manually simulate several simple attacks. For websites with low anomaly detection accuracy, we increase the proportion of their related traffic in the training samples to improve the performance on the validation set.

[0095] The workload optimization was necessary because the company's hardware environment was limited, and GPUs were not available in the actual deployment environment. Therefore, for the second and third anomaly detection models mentioned in this application, the model structure needed to be continuously simplified without affecting accuracy. This required repeated optimization of the model structure, including reducing the number of neural network layers and the scale of parameters, ultimately achieving acceptable running efficiency.

[0096] Regarding offline training performance, after optimization, all three methods successfully detected our manually simulated attacks. The Effect table lists the performance of the three methods before and after optimization on a validation set composed of normal data.

[0097] The accuracy of the methods is as follows. It can be seen that Method 1 and Method 2 have fewer false positives for normal requests, while Method 3 misclassifies many normal requests as abnormal. After optimization, the accuracy of the three methods on the validation set is as follows:

[0098] It has increased significantly.

[0099] Validation set accuracy before tuning Validation set accuracy after optimization Method 1 98.8％ 99.3％ Method 2 97.2％ 99.0％ Method 3 85.6％ 93.1％

[0100] Table 2: Effects of offline training.

[0101] This application discloses how to optimize the three anomaly detection models mentioned above to improve detection accuracy and adapt them to limited hardware resources through model optimization.

[0102] After offline training, the trained web anomaly detection system was deployed to the company's intranet. During a 14-day attack and defense training exercise, it monitored business traffic in a bypass mode and issued online alerts.

[0103] The web anomaly detection system was deployed as a Docker container, configured with 2 cores and 64GB of RAM. The web anomaly detection system (SeqTrident) maintained stable operation for 14 days. During peak business periods, SeqTrident alarms experienced a delay of up to 10 minutes due to the processing speed lagging behind the request reception speed. However, during periods of low business volume, such as at night, SeqTrident was generally able to catch up with request reception.

[0104] Compared with the algorithms from other security vendors deployed by the company, the web anomaly detection system in this application detected a total of 19 attack attempts that had not been detected by other security vendors, all of which were manually confirmed. Figure 4 This is a bar chart showing the number of additional alarms discovered in one embodiment of this application, i.e., the number of additional alarms (attack attempts not identified by other vendors) discovered each day during the 14-day training period by SeqTrident.

[0105] This web anomaly detection system and method combines three different sequential anomaly detection models to effectively detect web attacks. The application uses a probabilistic segmentation-based AC automaton for request sequence modeling, a Transformer-based reconstruction model for request sequence reconstruction, and an LSTM-based model for request sequence prediction. In practice, it was deployed in a large enterprise and detected 17 attacks that other traditional algorithms failed to detect during a 14-day large-scale attack and defense exercise. Experimental results show that the average detection accuracy of the web anomaly detection system and method is 96.7%, proving that the proposed system and method can effectively detect web attacks.

[0106] The web anomaly detection system and method of this application employ a model capable of actively learning from logs containing only normal behavior and extracting sequence information without requiring specific API documentation. This application uses a data preprocessing module to transform network traffic into an APIID sequence that can be used as input by the anomaly detection model.

[0107] This application addresses the problem of unsatisfactory detection results when using a single model, and achieves better detection results by combining the advantages of three different anomaly detection models.

[0108] This application employs a method for optimizing anomaly detection models, and describes how to fine-tune the models to improve detection accuracy and adapt them to limited hardware resources.

[0109] The embodiments of the web anomaly detection system described above are basically similar to the method embodiments, so the description is relatively simple. For relevant details, please refer to the description of the method embodiments.

[0110] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0111] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0112] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A WEB anomaly detection method, characterized by, include: The historical web request records of different users are categorized into different API interfaces, and the numerical values ​​of different API interfaces are encoded into API ID sequences; Create multiple anomaly detection models based on the API ID sequence to detect abnormal web requests; Each of the aforementioned anomaly detection models is used to detect whether the target web request is an abnormal request. An anomaly detection model generates an alert if the detection result indicates an abnormal request. By statistically analyzing whether the number of anomaly detection models that issue alarms meets a preset threshold, it is determined whether to raise the risk level of the IP address of the web request. When the number of anomaly detection models that issue alarms triggered by the same IP address reaches the preset threshold, the risk level of the IP address is raised. Categorizing users’ historical web requests into different API interfaces includes: analyzing and identifying the API corresponding to a web request using parameter elimination; The URLs obtained after parameter elimination are categorized, with each URL corresponding to an API ID; The creation of multiple anomaly detection models based on the API ID sequence for detecting abnormal web requests includes: The API ID sequence is divided into multiple substrings, each substring corresponding to a web request; An AC automaton is built based on multiple substrings to serve as an anomaly detection model for identifying abnormal web requests; The anomaly detection model is a Transformer model. The API ID sequence is input into the Transformer model to obtain the reconstructed API ID. The abnormal web request is identified by the deviation between the API ID sequence before and after reconstruction. The anomaly detection model is an LSTM Predictor model. By inputting the API ID sequence given by the LSTM Predictor model, it outputs the next API ID. If the probability of the next API ID meets a predetermined threshold, the web request is normal; otherwise, the web request is an abnormal web request.

2. The WEB anomaly detection method of claim 1, wherein, The parameter elimination includes: Remove query parameters starting with "?" from the URL to identify whether they are similar API interfaces; or Remove long strings or numbers exceeding a specified threshold from a URL's path at a certain level and replace them with appropriate placeholders to identify whether they belong to the same type of API; or Remove static filenames from the last-level path of the URL, retaining only the file extension, to identify whether they are similar API interfaces; or Each distinct path in the URL is treated as a path parameter and replaced with a placeholder to identify whether they are API interfaces of the same type.

3. The WEB anomaly detection method of claim 1, wherein, The creation of multiple anomaly detection models based on the API ID sequence for detecting abnormal web requests includes: Attach a pointer to a node on the AC automaton to each user's session; If the current node of a web request moves to the root node, the web request does not match any of the substrings and is identified as an abnormal web request.

4. A WEB anomaly detection system characterized by, include: The data preprocessing module is used to classify the historical web request records of different users into different API interfaces and encode the different API interface values ​​into API ID sequences; An anomaly detection model creation module is used to create multiple anomaly detection models for detecting abnormal web requests based on the API ID sequence; The abnormal request alarm module generates alarms when the detection result is an abnormal request, based on the anomaly detection model. The alarm processing module determines whether to raise the risk level of the IP address of the web request by counting whether the number of anomaly detection models of the issued alarms meets the preset threshold. When the number of anomaly detection models of the same IP address triggering alarms reaches the preset threshold, the risk level of the IP address is raised. The API identification module analyzes and identifies the API corresponding to a web request by using parameter elimination. The API identification module also includes classifying URLs obtained after parameter elimination, with each URL corresponding to an API ID; The anomaly detection model creation module includes: The API ID sequence is divided into multiple substrings, each substring corresponding to a web request; An AC automaton is built based on multiple substrings to serve as an anomaly detection model for identifying abnormal web requests; The anomaly detection model is a Transformer model. The API ID sequence is input into the Transformer model to obtain the reconstructed API ID. The abnormal web request is identified by the deviation between the API ID sequence before and after reconstruction. The anomaly detection model is an LSTM Predictor model. By inputting the API ID sequence given by the LSTM Predictor model, it outputs the next API ID. If the probability of the next API ID meets a predetermined threshold, the web request is normal; otherwise, the web request is an abnormal web request.

5. The WEB anomaly detection system as described in claim 4, characterized in that, The parameter elimination includes: Remove query parameters starting with "?" from the URL to identify whether they are similar API interfaces; or Remove long strings or numbers exceeding a specified threshold from a URL's path at a certain level and replace them with appropriate placeholders to identify whether they belong to the same type of API; or Remove static filenames from the last-level path of the URL, retaining only the file extension, to identify whether they are similar API interfaces; or Each distinct path in the URL is treated as a path parameter and replaced with a placeholder to identify whether they are API interfaces of the same type.

Images