Method for blocking abnormal access of website service and computer program product

Abstract

The invention provides a method for blocking abnormal access of website service and a computer program product. The method comprises the following steps of: collecting all external accesses, HTTP header data and a processing state of a servo host within a period of time before a website service officially starts to operate, classifying the accesses, and taking out non-secure accesses to construct a judgment model; the judgment model is an intelligent model established by a machine learning algorithm, and unreliable access can be filtered out in non-secure access when website service is officially operated, so that the quality of the website service is improved, and external malicious damage is avoided.

Description

Method and computer program product for blocking abnormal access to website services TECHNICAL FIELD

[0001] The present invention relates to a web application firewall, in particular, a method and computer program product for blocking abnormal access to website services. BACKGROUND

[0002] Servers deploying web applications are the technical architecture of a web platform to provide specific services. Based on business factors, these web platforms are often subjected to many disturbances or attacks, such as bot attacks, distributed denial-of-service attacks, cross-site scripting attacks, SQL injection, etc., causing a significant drop in service quality, and even server crashes. To solve these problems, many technology providers have developed different web application firewalls to cope with them. These web application firewalls can help web platforms establish and manage rules to protect against Internet threats. Their technology is mainly directed at IP addresses, HTTP headers, HTTP bodies, URI strings, cross-site command files, SQL injection attacks, and other Open Web Application Security Project (OWASP) defined vulnerabilities, to build and deploy applications to protect web platforms, while collecting logs to meet compatibility and analysis needs. TECHNICAL PROBLEM

[0003] As the web application firewall technology improves, the initiators of these network disturbances or attacks also have increasingly improved improvements. Although both sides are constantly growing, for the network platform operators who are victims, there are always some difficult-to-express problems that cannot be tolerated. One of the main reasons for these malicious behaviors that cannot be eradicated or suppressed is that web application firewall technology providers usually propose solutions to attacks in a passive manner, and then deploy these temporary solutions to servers from a remote location. This approach tests the technical ability of engineers, but it is difficult to say that it is effective at the moment. On the other hand, with the rapid development of artificial intelligence software and hardware, using artificial intelligence to track and learn the results of a particular technology is often better than manually operating the technology. Therefore, introducing artificial intelligence into web application firewall technology has become a new direction for research and development. The present invention is developed under this idea, and a method and computer program product for blocking abnormal access to website services are proposed. TECHNICAL SOLUTION

[0004] This paragraph extracts and compiles some features of the present invention. Other features will be disclosed in subsequent paragraphs. The purpose is to cover the spirit and scope of the additional patent application range, various modifications and similar arrangements.

[0005] The present application provides a method for blocking abnormal access of HTTP service of a website, which is executed by a server host providing HTTP service of the website, and includes the following steps: S01, continuously recording a plurality of accesses based on HTTP and HTTP status codes processed by the server host for each access within a sampling time, wherein the accesses corresponding to HTTP status codes of 1xx message, 2xx success or 3xx redirection are normal connections; S02, taking a uniform resource identifier (URI) and an HTTP method in each access to form a feature pair; S03, distinguishing each access as a secure access or a non-secure access based on the feature pairs, wherein the secure access has target feature pairs sequentially meeting the following conditions: 1) the number of normal connections of the access using the target feature pairs exceeds a threshold value; 2) the number of accesses using the target feature pairs exceeds the sum of the third quartile and 1.5 times the interquartile range obtained by quartile statistics of the number of accesses using all feature pairs; and 3) the proportion of the number of normal connections of the access using the target feature pairs to the total number of accesses using the target feature pairs is not less than 95%; S04, taking a combination of different feature pairs, remote users, original locations (X-Forwarded-For) and HTTP reference addresses in non-secure accesses as feature inputs, and whether the non-secure accesses are normal connections as learning judgment values, constructing a judgment model by an eXtreme Gradient Boosting (XGBoost) machine learning algorithm, wherein the judgment model can calculate corresponding judgment values for new input non-secure accesses to determine whether the non-secure accesses are normal connections; and S05, after the sampling time, allowing the server host to respond to the secure accesses and the accesses with judgment values of normal connections obtained by the judgment model.

[0006] According to the present application, a step a1) of deleting accesses from a specific IP in the server host and exceeding an upper limit number of times per unit time and HTTP status codes of the accesses can be further included between step S01 and step S02.

[0007] According to the present application, a step S041 of excluding accesses including specific parameters from the secure accesses can be further included between step S04 and step S05.

[0008] According to the present application, a step S042 of excluding accesses including specific cross-site instruction codes from the secure accesses can be further included between step S04 and step S05.

[0009] According to the present application, a step S043 of excluding accesses including program codes from the secure accesses can be further included between step S04 and step S05.

[0010] In an embodiment, if any non-secure access occurs more than once, the non-secure access is normal connection if the number of times of occurrence of the non-secure access is more than one.

[0011] According to the present application, the string length can be further added to the feature input in step S04.

[0012] Finally, the URI can be further processed by natural language processing and Murmur hash in step S04 to generate feature labels for constructing the judgment model. The natural language processing can be term frequency-inverse document frequency (TF-IDF).

[0013] The present application also provides a computer program product for blocking non-normal access of a website HTTP service. The computer program product is installed in a server host and includes a plurality of program instructions for executing the method described above. The method of the present application collects all external accesses, HTTP header information and processing states of the server host within a period of time before the website service formally starts operation, classifies the accesses, and extracts non-secure accesses to construct the judgment model. The judgment model is an intelligent model established by a machine learning algorithm, which can filter out unreliable accesses among non-secure accesses when the website service formally starts operation, improve the quality of the website service, and avoid malicious damage from the outside. BRIEF DESCRIPTION OF DRAWINGS

[0014] FIG. 1: Application scenario of a method for blocking non-normal access of a website HTTP service according to the present application;

[0015] FIG. 2: Flowchart of the implementation steps of the method for blocking non-normal access of a website HTTP service;

[0016] FIG. 3: Partial feature pairs, their HTTP status codes and access judgments at the beginning of the sampling time;

[0017] FIG. 4: Display of the number of accesses corresponding to the feature pairs with threshold values greater than 1.

[0018] REFERENCE NUMERALS

[0019] 1. A smart phone;

[0020] 2. A desktop computer;

[0021] 3. A notebook computer;

[0022] 10. A server host;

[0023] 11. A web application;

[0024] 12. A computer program product;

[0025] 20. A proxy server;

[0026] 30. Malicious server;

[0027] N. Network. Best mode of the invention

[0028] The present invention will be described more specifically by referring to the following embodiments.

[0029] Please see FIG. 1, which illustrates the application scenario of a method for blocking abnormal access to HTTP service of a website (hereinafter referred to as the present method), which is executed by a server host 10 providing HTTP service of a website. Here, the server host 10 can represent a server cluster, in which a network application program 11 driving the service of a website (specifically, online service for information transmission through HTTP, such as the vehicle buying and selling information providing service in the present embodiment) is installed, and a computer program product 12 for executing the present method is also installed. More specifically, the computer program product 12 includes a plurality of program instructions for executing the specific operation steps of the present method as will be described below, and the type of each program instruction is not limited.

[0030] In the external connection state of the server host 10, only a plurality of common devices are proposed in FIG. 1 to illustrate. A user A uses a smart phone 1 to directly connect with the server host 10 through a network N, sends an HTTP request to the server host 10, and receives an HTTP response from the server host 10. In this interactive process, the server host 10 can obtain the IP information of the smart phone 1, such as the IP address, the Internet service provider, the IP host name, etc. A user B operates a desktop computer 2 to connect with the server host 10 through a proxy server 20, and can also use the aforementioned online service. However, in the interactive process, the IP information obtained by the server host 10 is about the proxy server 20, which is beneficial to protect the privacy of the user B. The operations of the user A and the user B are normal and acceptable to the website service operator. A user C is a car market explorer who needs to know the latest price information of buying and selling vehicles in real time, and therefore operates a notebook computer 3 to directly connect with the server host 10 through the network N to continuously request information from the server host 10 by using a crawler software. Since the website service is for the public, the needs of each end user are different, and in theory, the website service operator should not refuse any request. However, this continuous transmission of data to a specific terminal consumes the resources of the server host 10, resulting in a decline in the overall service quality, and the server host 10 wants to avoid receiving access from the IP of the notebook computer 3. A malicious server 30 is a competitor of the website service operator, and its purpose of directly connecting with the server host 10 through the network N is to implant a malicious program in the server host 10 to obtain some protected commercial secrets in the database of the server host 10. The computer program product 12 based on the method is to block the access from the notebook computer 3 and the malicious server 30.

[0031] Please see Fig. 2, which is a flow chart of the implementation steps of the present method. The first step of the present method is S01: continuously record HTTP-based accesses and the HTTP status code of each access handled by the server host within a sampling time, wherein the HTTP status code is 1xx information, 2xx success, or 3xx redirection corresponding access is normal connection. The sampling time is the time for collecting data, and the data within the time is used to establish a secure access and a machine learning model, which can be immediately operated online after establishment. The sampling time is determined according to the architecture of the network application program 11 and the overall size of the network service; it can be as short as several hours, as long as several weeks, or even several months. Since the server host 10 will issue an HTTP response according to the actual situation after receiving an HTTP request from the outside, the HTTP header in the HTTP response carries the HTTP status code to reply to the request end request processing status. All HTTP status codes are divided into five categories: 1xx information, 2xx success, 3xx redirection, 4xx client error, and 5xx server error. According to the present application, the access with the first three HTTP status codes is defined as normal connection, and the access with the last two HTTP status codes is defined as abnormal connection.

[0032] The second step of the present method is S02: taking out the uniform resource identifier (URI) and the HTTP method in each access to form a feature pair. For a better understanding, please see Fig. 3, which is a table listing part of the feature pairs, their HTTP status codes, and access judgments at the beginning of the sampling time. The HTTP method includes OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, CONNECT, and PATCH. According to the time of occurrence, the different combinations of URI and HTTP method (feature pairs) are numbered from 1, and the same feature pair number remains unchanged, such as www.car.com.tw / and GET of the No. 4 feature pair. Only the first 20 feature pairs are shown in Fig. 3, and there are still many feature pairs before the end of the sampling time. The HTTP status code in the HTTP response of the server host 10 corresponding to each requested URI and HTTP method is also listed in the table. There are three HTTP status codes, 499, 404, and 499, whose accesses belong to abnormal connection, and the rest of the accesses are normal connection.

[0033] The third step of the method is S03: distinguishing each access as a safe access or a non-safe access based on the feature pairs. A safe access has a target feature pair that sequentially meets the following conditions: 1) the number of normal connections of the access using the target feature pair exceeds a threshold value; 2) the number of accesses using the target feature pair exceeds the sum of the third quartile (Q3) and 1.5 times the interquartile range (Q3-Q1) of the quartile statistics of the distribution of the number of accesses using all feature pairs; and 3) the proportion of the number of normal connections of the access using the target feature pair to the total number of accesses using the target feature pair is not less than 95%. In this step, the safe access must sequentially and completely meet the three screening methods. The target feature pair that meets the conditions is a safe access, and the rest is a non-safe access.

[0034] The first screening method: the number of normal connections of the access using the target feature pair exceeds a threshold value. For example, the threshold value is set to N times. If the number of normal connections of the feature pair (www.car.com.tw / images / 888.png, GET) exceeds N times within the sampling time, (www.car.com.tw / images / 888.png, GET) is a target feature pair. These feature pairs can enter the next screening.

[0035] The second screening method involves using the number of visits to the target feature pair. If this number exceeds the sum of the third quartile (Q3) obtained by performing quartile analysis on the distribution of visits to all feature pairs, and the interquartile range (Q3-Q1) is 1.5 times the quartile distance, then feature pairs are identified using the upper limit of the interquartile range. All feature pairs are analyzed using quartile analysis to find the sum of the third quartile and the interquartile range of 1.5 times the quartile distance. An upper limit is defined based on this; outliers exceeding this limit are the corresponding hot feature pairs, and these feature pairs can proceed to the next screening step. For a better understanding, please see Figure 4, which shows the access count for all features with a threshold value greater than 1. The feature pairs are arranged from left to right according to the number of accesses (the access counts for the feature pairs on the horizontal axis from left to right are 460004, 348682, 228399, 155309, 151769, 42881, 40394, 18023, 1274, 806, 668, 417, 289, 275, 242, 238, 237, 236, 235, 233, 137, 86, 72, 65, 35, 24, 18, 14, 11, 9, 8, 7, 6, 6, 4, 4, 4, 3, 3, 3, 3, 3, 2 ... For example, the feature pair with the most visits is feature pair #1 (460,004 visits). After quartile analysis, the feature pair for the third quartile is feature pair #15, with 242 visits; the feature pair for the first quartile is feature pair #44, with 2 visits. In this case, the sum of the third quartile and 1.5 times the interquartile range is 242 + 1.5. (242-2)≈602 (rounded to the nearest whole number). Feature pairs with a single feature pair that have been accessed more than 602 times are the target feature pairs. If a feature pair has this feature pair, it can proceed to the next screening step.

[0036] The third screening method: The ratio of the number of normal connections to the total number of accesses using the target feature pair is not less than 95%. For example, if the total number of accesses with the feature pair (www.car.com.tw / ccu / car-iinfor.txt, GET) is 673, and 668 of them are normal connections, the ratio of normal connections to the total number is 99.2%, which is greater than 95%. Therefore, this feature pair is the target feature pair, and accesses with these target feature pairs are considered safe accesses.

[0037] After distinguishing the secure access from the non-secure access, the fourth step of the method is to construct a judgment model by taking out some specific feature pairs in the non-secure access and adding relevant information. The specific content is S04: taking the combination of different feature pairs, remote user, original location (X-Forwarded-For) and HTTP reference address in the non-secure access as feature input, and whether the aforementioned non-secure access is a normal connection as a learning judgment value, a judgment model is constructed by eXtreme Gradient Boosting (XGBoost) machine learning algorithm. The judgment model can calculate the corresponding judgment value of the new input non-secure access to determine whether the non-secure access is a normal connection. Remote user, original location and HTTP reference address are the HTTP header contents of each access, and the non-secure access as a normal connection can be set to 0 and the non-secure access as an abnormal connection can be set to 1 as a judgment value. The characteristics of XGBoost machine learning algorithm are high accuracy, fast training speed, easy to implement on a single machine, and best applied to the scene of the present application. An example of an input is (www.car.com.tw / ccu / car-sell / upert, GET), johoned (remote user), X-Forwarded-For: 130.78.138.66, 129.78.64.110, http: / / we.cartrading.com / Main_Page, 1. In this embodiment, the judgment value obtained by the judgment model is between 0 and 1, and 0.5 is the boundary. The judgment value less than 0.5 indicates that the non-secure access is a normal connection, and the judgment value greater than or equal to 0.5 indicates that the non-secure access is an abnormal connection.

[0038] Finally, the fifth step of the method is S05: after the sampling time, the server host 10 responds to some secure accesses and accesses with a judgment value of normal connection obtained by the judgment model. That is, when the computer program product 12 is normally operating, only the accesses previously judged as secure accesses and the accesses with a judgment value of normal connection obtained by the judgment model will be executed by the network application program 11 to respond to the requests of these accesses, and the remaining accesses will be discarded by the server host 10 without processing, so as to avoid resource waste and prevent the website service from being systematically damaged.

[0039] According to the present application, a step S011 can be further included between step S01 and step S02, which is to delete the accesses from a specific IP in the server host 10 that exceed a threshold number of times per unit time, and the HTTP status codes of these accesses. For example, the accesses from IP 132.128.64.33 are very frequent, which exceed the threshold number of 60 times per minute, and the accesses from this IP can be a crawler software or a malicious attack, and thus are excluded from the acceptable accesses and do not enter the subsequent machine learning to establish the judgment model, so as to avoid the judgment model misjudgment and make the accesses from this IP acceptable.

[0040] In addition, the present method can further include a step S041 between step S04 and step S05, which is to exclude the accesses including a specific parameter from the secure accesses. For example, the accesses with the parameter?getfree5452 are found to be a factor that causes the network application 11 to become poor, and thus the accesses with the same parameter are excluded from the secure accesses, and the server host 10 does not respond to these deleted accesses.

[0041] The present method can further include a step S042 between step S04 and step S05, which is to exclude the accesses including a specific cross-site scripting code from the secure accesses. Cross-site scripting (XSS) is a security vulnerability that allows intruders to insert program codes into the network application 11 by using the website, and this attack involves a web server that dynamically generates HTML pages, such as from a malicious server 30. However, cross-site scripting is not purely intended to be malicious, and thus the accesses including a specific cross-site scripting code that has been found to be malicious are excluded from the secure accesses, and the server host 10 does not respond to these deleted accesses.

[0042] The present method can further include a step S043 between step S04 and step S05, which is to exclude the accesses including program codes from the secure accesses. This method is more aggressive and effective than step S042: the server host 10 does not respond to any accesses with program codes.

[0043] In one embodiment, if any one of the non-secure accesses occurs more than once, i.e., not by chance, then the learning judgment value is normal connection. This greatly limits the non-secure access, and the judgment model can "whitewash" to obtain the possibility of the judgment value being normal connection. For example, if any access is listed as a non-secure access due to the number of occurrences, unless all connections of this access are normal connections (such as no connection failure of the server host 10 or other problems), it is possible to set the judgment value as normal connection, so that the judgment model will determine the judgment value of the same access in the future as normal connection, and the server host 10 will respond.

[0044] In addition, the string length can be further added to the feature input in step S04. In this way, the string length will also be a basis for the judgment model to determine the judgment value when executing. In order to make the XGBoost machine learning algorithm more effective, the URI in step S04 is subjected to natural language processing and Murmur hash to generate feature labels for constructing the judgment model. Natural language processing is a branch of artificial intelligence that focuses on processing and understanding human language. The term frequency-inverse document frequency (TF-IDF) is used to evaluate the importance of a word to a file set or a document in a corpus. Since the composition of the URI also contains human-understandable text, the importance of the word is obtained by performing TF-IDF conversion on these texts, and then the degree of influence of non-secure access is extracted. Murmur hash is a non-encrypted hash function suitable for general hash-based lookup. The URI subjected to natural language processing is subjected to Murmur hash operation, and the obtained feature label is used as feature input, which can accelerate the effect of the XGBoost machine learning algorithm when constructing the judgment model.

[0045] Although the present application has been disclosed as above, it is not intended to limit the present application, and any person skilled in the art can make some changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of the claims.

Claims

1. A method for blocking abnormal access to a website service, executed by a server host providing HTTP service of the website, characterized in that, The method comprises steps of: S01: continuously recording HTTP-based accesses and HTTP status codes processed by the server host for each access within a sampling time, wherein the HTTP status codes are 1xx messages, 2xx successes, or 3xx redirections corresponding to normal connections; S02: extracting uniform resource indicators and HTTP methods in each access to form a feature pair; S03: distinguishing each access as a secure access or a non-secure access based on the feature pairs, wherein the secure access has target feature pairs that sequentially satisfy the following conditions: 1) the number of normal connections of the access using the target feature pair exceeds a threshold value; 2) the number of accesses using the target feature pair exceeds the sum of the third quartile and 1.5 times the interquartile range obtained by quartile statistics of the distribution of the number of accesses using all feature pairs; and 3) the proportion of the number of normal connections of the access using the target feature pair to the total number of accesses using the target feature pair is not less than 95%; S04: using different combinations of feature pairs, remote users, original locations, and HTTP reference addresses in non-secure accesses as feature inputs, and whether the non-secure accesses are normal connections as learning judgment values, constructing a judgment model through an eXtreme Gradient Boosting machine learning algorithm, which can calculate corresponding judgment values for new input non-secure accesses to determine whether the non-secure accesses are normal connections; and S05: after the sampling time, allowing the server host to respond to the secure accesses and the accesses with judgment values as normal connections obtained through the judgment model. Between step S01 and step S02, a step S011 is included: deleting accesses from specific IPs in the server host that exceed an upper limit number of times per unit time, and the HTTP status codes of the accesses.

2. The method of blocking abnormal access to a web site service of claim 1, wherein, Between step S04 and step S05, a step S041 is included: excluding accesses including specific parameters from the secure accesses.

3. The method of blocking abnormal access to a web site service of claim 1, wherein, Between step S04 and step S05, a step S042 is included: excluding accesses including specific cross-site instruction codes from the secure accesses.

4. The method of blocking abnormal access to a web site service of claim 1, wherein, Between step S04 and step S05, a step S043 is included: excluding accesses including program codes from the secure accesses.

5. The method of blocking abnormal access to a web site service of claim 1, wherein, When the number of any non-secure access exceeds one, the learning judgment value is normal connection only when the non-secure access is normal connection.

6. The method of blocking abnormal access to a web site service of claim 1, wherein, In step S04, the length of the string is further added to the feature input.

7. The method of blocking abnormal access to a web site service of claim 1, wherein, In step S04, the URI is subjected to natural language processing and Murmur hashing to generate feature labels for constructing the judgment model.

8. The method of blocking abnormal access to a web site service of claim 1, wherein, The natural language processing is term frequency-inverse document frequency.

9. The method of blocking abnormal access to a web site service of claim 8, wherein, Installed in a server host, comprising a plurality of program instructions to execute the method of any one of claims 1-9.

10. A computer program product for blocking abnormal access to HTTP services of a website, characterized in that, ​

Images