A web application firewall system based on hidden markov model

Abstract

The application discloses a kind of Web application firewall systems based on hidden Markov model, including preprocessing module, for receiving client through browser to server initiated request analysis and preprocessing, obtain the result after serialization is saved and is handed over to learning module learning, learning module, for obtaining the attribute field of HTTP request from preprocessing module, the operation of judging field type is carried out, and the appropriate model is selected from three learning models of HMM and is learned, and generates learning result;Detection module.The learning module of the application is analyzed by HTTP Request data, and is preprocessed according to its different properties, and different features are extracted to form sequence, non-negative integer and character, and the result of learning will become the basis for the system to judge whether Web request is abnormal, in addition to detecting general attacks, potential Web attacks can also be detected, so as to achieve the purpose of protecting Web security.

Description

Technical Field

[0001] This invention relates to the field of firewall system technology, specifically to a web application firewall system based on a hidden Markov model. Background Technology

[0002] With the rapid development of information technology, Web services have become an important carrier of the Internet industry. However, numerous Web security vulnerabilities have also emerged, making the protection of Web services from attacks a hot research topic in the security field. Traditional network security devices primarily rely on packet inspection, operating at the network and transport layers of the OSI model, and cannot effectively protect the Web at the application layer. In recent years, Web application firewalls based on application layer defense have appeared on the market.

[0003] A firewall system is a combination of components placed between different networks (such as a trusted corporate intranet and an untrusted public network) or network security domains. It protects the network by monitoring, restricting, and modifying data flows across the firewall, shielding the network's internal information, structure, and operational status from the outside world as much as possible. Logically, it acts as a separator, a limiter, and an analyzer, effectively monitoring any activity between the intranet and the Internet, ensuring the security of the internal network.

[0004] Existing web application firewalls are mostly based on blacklists and whitelists with known rule bases. They inspect data entering and leaving web servers but cannot detect potential or unknown web attacks. Therefore, it is necessary to design a web application firewall system based on a hidden Markov model. Summary of the Invention

[0005] To address the above issues and overcome the shortcomings of existing technologies, this invention provides a Web application firewall system based on a Hidden Markov Model. The learning module analyzes HTTP request data, preprocesses it according to its different attributes, and extracts different features to form sequences, non-negative integers, and characters. The learning results become the basis for the system to judge whether a Web request is abnormal. In addition to detecting general attacks, it can also detect potential Web attacks, thereby achieving the purpose of protecting Web security.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a Web application firewall system based on a Hidden Markov Model, comprising...

[0007] The preprocessing module is used to receive requests from the client through the browser to the server, analyze and preprocess them, and save the serialized results before delivering them to the learning module for learning.

[0008] The learning module is used to determine the field type after obtaining the attribute fields of the HTTP request from the preprocessing module, select the appropriate model from the three learning models of HMM for learning, and generate learning results.

[0009] The detection module, upon receiving a serialized web request, first parses the web request to obtain the attributes and key fields in the request, and then calls the learning module and the CRS rule base module to judge the web request;

[0010] The response module is used to receive the results from the detection module, determine the threat level, and select the appropriate response level to respond.

[0011] The CRS rule base module is used to judge whether a request is abnormal or not when the learning module initializes request data, and to mark it in order to obtain an accurate and reliable training set.

[0012] The logging module is used to record the processing and response information of the detection and response modules. The log information is stored in the form of a database.

[0013] Preferably, the workflow of the learning module is as follows:

[0014] 1) Obtain the HTTP request data from the preprocessing module to form a training set;

[0015] 2) Classify the training set data according to the three types of learning models;

[0016] 3) Extract the data from the training set and determine the data type. If the data type is a sequence, then pass it to the α model for processing; otherwise, continue the determination.

[0017] 4) If the result is determined to be a non-negative integer, then it is handled by the β model; otherwise, the process continues.

[0018] 5) If the data is determined to be of character type, it will be handled by the γ model; otherwise, the preprocessed data will be considered incorrect.

[0019] 6) Finally, save all the learning results.

[0020] Preferably, the α model utilizes the Baum-Welch training algorithm and the Viterbi algorithm of Hidden Markov Models to achieve machine learning and find the optimal sequence.

[0021] Preferably, the β model is used to process sets of non-negative integers such as length or number of characters in HTTP request attributes, and to make a determination using Chebyshev's inequality.

[0022] Preferably, the data processing flow of the β model is as follows:

[0023] 1) Retrieve the attribute field 'a' based on a non-negative integer value from the HTTP request. i ;

[0024] 2) Statistical attribute domain a i The length of each element in the string is X, which is the length of the generated string. i ;

[0025] 3) Calculate the string length X using Chebyshev's inequality. i Expected length μ and variance σ 2 ;

[0026] 4) Create another SX to store the size of these attribute values, and check the integer attributes of incoming and outgoing web requests.

[0027] Preferably, the γ model determines the probability of a character's occurrence by statistically analyzing the frequency of its occurrence.

[0028] Preferably, the specific process of the γ model is as follows:

[0029] Let the character training set S = {x1, x2, ..., x} n According to the ASCII table of 256 characters, the set S has 256 elements, each element corresponding to a character. The frequency of each character is calculated by counting the number of occurrences and then dividing by the total number of characters.

[0030] Preferably, the step of calling the learning module and the CRS rule base module to judge the Web request is as follows:

[0031] 1) First, use the learning results from the learning module to make a judgment and obtain the result of whether it is abnormal or not;

[0032] 2) Next, the CRS rule base module is called to perform the detection and obtain the result of whether it is abnormal or not.

[0033] The beneficial effects of this invention are as follows:

[0034] 1. The learning module analyzes HTTP request data, preprocesses it according to its different attributes, and extracts different features to form sequences, non-negative integers and characters. The learning results will become the basis for the system to judge whether the web request is abnormal. In addition to detecting general attacks, it can also detect potential web attacks, thereby achieving the purpose of protecting web security.

[0035] 2. The response module is configured to classify the severity levels of web attacks and then take different measures for different levels, such as dropping data packets or terminating services, in order to protect the system. Attached Figure Description

[0036] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0037] Figure 1 This is a block diagram of a Web application firewall system module based on a hidden Markov model according to the present invention.

[0038] Figure 2 This is a flowchart of the detection module processing of a Web application firewall system based on a hidden Markov model according to the present invention. Detailed Implementation

[0039] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0040] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0041] The technical solution of the present invention will now be clearly and completely described in conjunction with the accompanying drawings.

[0042] Depend on Figure 1 and Figure 2 The present invention provides the following technical solution: a Web application firewall system based on a Hidden Markov Model, comprising...

[0043] The preprocessing module is used to receive requests from the client through the browser to the server, analyze and preprocess them, and save the serialized results before delivering them to the learning module for learning.

[0044] The learning module is used to determine the field type after obtaining the attribute fields of the HTTP request from the preprocessing module, select the appropriate model from the three learning models of HMM for learning, and generate learning results.

[0045] The detection module, upon receiving a serialized web request, first parses the web request to obtain the attributes and key fields in the request, and then calls the learning module and the CRS rule base module to judge the web request;

[0046] The response module is used to receive the results from the detection module, determine the threat level, and select the appropriate response level to respond.

[0047] The CRS rule base module is used to judge whether a request is abnormal or not when the learning module initializes request data, and to mark it in order to obtain an accurate and reliable training set.

[0048] The logging module is used to record the processing and response information of the detection and response modules. The log information is stored in the form of a database.

[0049] Preferably, the workflow of the learning module is as follows:

[0050] 1) Obtain the HTTP request data from the preprocessing module to form a training set;

[0051] 2) Classify the training set data according to the three types of learning models;

[0052] 3) Extract the data from the training set and determine the data type. If the data type is a sequence, then pass it to the α model for processing; otherwise, continue the determination.

[0053] 4) If the result is determined to be a non-negative integer, then it is handled by the β model; otherwise, the process continues.

[0054] 5) If the data is determined to be of character type, it will be handled by the γ model; otherwise, the preprocessed data will be considered incorrect.

[0055] 6) Finally, save all the learning results.

[0056] Preferably, the α model utilizes the Baum-Welch training algorithm and the Viterbi algorithm of Hidden Markov Models to achieve machine learning and find the optimal sequence.

[0057] Preferably, the β model is used to handle sets of non-negative integers such as length or number of characters in HTTP request attributes, and Chebyshev's inequality is used for determination.

[0058] Preferably, the data processing flow for the β model is as follows:

[0059] 1) Retrieve the attribute field 'a' based on a non-negative integer value from the HTTP request. i ;

[0060] 2) Statistical attribute domain a iThe length of each element in the string is X, which is the length of the generated string. i ;

[0061] 3) Calculate the string length X using Chebyshev's inequality. i Expected length μ and variance σ 2 ;

[0062] 4) Create another SX to store the size of these attribute values, and check the integer attributes of incoming and outgoing web requests.

[0063] Preferably, the γ model determines the probability of a character's occurrence by statistically analyzing the frequency of its occurrence.

[0064] Preferably, the specific process of the γ model is as follows:

[0065] Let the character training set S = {x1, x2, ..., x} n According to the ASCII table of 256 characters, the set S has 256 elements, each element corresponding to a character. The frequency of each character is calculated by counting the number of occurrences and then dividing by the total number of characters.

[0066] Preferably, the steps for judging Web requests by calling the learning module and the CRS rule base module are as follows:

[0067] 1) First, use the learning results from the learning module to make a judgment and obtain the result of whether it is abnormal or not;

[0068] 2) Next, the CRS rule base module is called to perform the detection and obtain the result of whether it is abnormal or not.

[0069] Working principle:

[0070] 1. The learning module analyzes HTTP request data, preprocesses it according to its different attributes, and extracts different features to form sequences, non-negative integers and characters. The learning results will become the basis for the system to judge whether the web request is abnormal. In addition to detecting general attacks, it can also detect potential web attacks, thereby achieving the purpose of protecting web security.

[0071] 2. The response module is configured to classify the severity levels of web attacks and then take different measures for different levels, such as dropping data packets or terminating services, in order to protect the system.

[0072] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A Web application firewall system based on a hidden Markov model, characterized in that: include The preprocessing module is used to receive requests from the client through the browser to the server, analyze and preprocess them, and save the serialized results before delivering them to the learning module for learning. The learning module is used to determine the field type after obtaining the attribute fields of the HTTP request from the preprocessing module, select an appropriate model from the three learning models of the HMM for learning, and generate learning results. The workflow for selecting an appropriate model from the three learning models of the HMM is as follows: 1) Obtain the HTTP request data from the preprocessing module to form a training set; 2) Classify the training set data according to the three types of learning models; 3) Extract the data from the training set and determine the data type. If the data type is a sequence, then pass it to the α model for processing; otherwise, continue the determination. 4) If the result is determined to be a non-negative integer, then it is handled by the β model; otherwise, the process continues. 5) If the data is determined to be of character type, it will be handled by the γ model; otherwise, the preprocessed data will be considered incorrect. 6) Finally, save all the learning results; The detection module, upon receiving a serialized web request, first parses the web request to obtain the attributes and key fields in the request, and then calls the learning module and the CRS rule base module to judge the web request; The response module is used to receive the results from the detection module, determine the threat level, and select the appropriate response level to respond. The CRS rule base module is used to judge whether a request is abnormal or not when the learning module initializes request data, and to mark it in order to obtain an accurate and reliable training set. The logging module is used to record the processing and response information of the detection and response modules. The log information is stored in the form of a database.

2. The Web application firewall system based on a hidden Markov model according to claim 1, characterized in that: The α model utilizes the Baum-Welch training algorithm and the Viterbi algorithm of Hidden Markov Models to achieve machine learning and find the optimal sequence.

3. A Web application firewall system based on a hidden Markov model according to claim 1, characterized in that: The β model is used to process the set of non-negative integers for length or number of characters in HTTP request attributes, and to make a determination using Chebyshev's inequality.

4. A Web application firewall system based on a hidden Markov model according to claim 3, characterized in that: The data processing flow for the β model is as follows: 1) Retrieve the attribute field 'a' based on a non-negative integer value from the HTTP request. i ; 2) Statistical attribute domain a i The length of each element in the string is X, which is the length of the generated string. i ; 3) Calculate the string length X using Chebyshev's inequality. i Expected length μ and variance σ 2 ; 4) Create another SX to store the size of these attribute values, and check the integer attributes of incoming and outgoing web requests.

5. A Web application firewall system based on a hidden Markov model according to claim 1, characterized in that: The γ model determines the probability of a character's occurrence by statistically analyzing its frequency.

6. A Web application firewall system based on a hidden Markov model according to claim 5, characterized in that: The specific process of the γ model is as follows: Let the character training set be S = {x1, x2, ..., xn}. n According to the ASCII table of 256 characters, the set S has 256 elements, each element corresponding to a character. The frequency of each character is calculated by counting the number of occurrences and then dividing by the total number of characters.

7. A Web application firewall system based on a hidden Markov model according to claim 1, characterized in that: The steps for judging Web requests by calling the learning module and the CRS rule base module are as follows: 1) First, use the learning results from the learning module to make a judgment and obtain the result of whether it is abnormal or not; 2) Next, the CRS rule base module is called to perform the detection and obtain the result of whether it is abnormal or not.

Images