Method and system for identifying abnormal traffic based on Openresty and linking iptables to perform IP blocking
By identifying abnormal traffic in openresty and linking iptables to perform IP blocking, the defense vacancy caused by the lack of linkage between openresty and iptables in the existing technology is solved, and stronger defense capabilities and higher processing efficiency are achieved.
Patent Information
- Application Number
- CN202311280529.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-28
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2043-09-28
AI Technical Summary
In the prior art, openresty and iptables have no linkage, which causes iptables to be unable to accurately identify malicious traffic when facing malicious scanning and sniffing, which may be bypassed, resulting in a defense gap.
By identifying abnormal traffic in openresty, and using the linkage blocking determination module to link with iptables, subscribe to redis information in real time, and determine whether IP blocking operation is performed according to the priority of the basic rules.
The linkage between openresty and iptables is realized, the defense capability is enhanced, the defense vacancy caused by relying solely on one side is avoided, the processing efficiency and traffic processing capacity is improved, and the manual maintenance cost is reduced.
Smart Images
Figure CN117354000B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer technology, and in particular relates to a method and system for identifying abnormal traffic based on Openresty and linking iptables to perform IP blocking. Background Art
[0002] At present, there are two main solutions for website system defense. One is to rely on iptables firewall for protection at the system level. iptables is a tool for configuring network packet filtering rules for Linux systems. It is a subsystem in the Linux kernel and is mainly used to implement firewall functions at the network layer and transport layer. It can check incoming and outgoing network packets and decide whether to allow or block the transmission of these packets based on predefined rule sets. For example, matching and filtering based on conditions such as source IP address, destination IP address, port number, protocol type, etc., helps protect Linux systems from malicious network attacks and unnecessary access by providing flexible and powerful network packet filtering and firewall functions. In iptables, "input" and "prerouting" are the names of two different tables (Table) and chains (Chain), which are used to define the processing rules of network packets at different stages.
[0003] Another type is the web application layer, which uses the waf (web application firewall) system implemented based on openresty for protection. It relies on lua to write related rules for interception, such as returning 403, 404 and other abnormal status codes after detecting an exception. Openresty is a high-performance web application based on the standard nginx core. It can extend many third-party modules. Web developers can use the lua programming language to program the core and various c modules. Simply put, it is equivalent to a web application platform that encapsulates nginx and integrates lua scripts. Waf is a security tool used to protect websites and web applications from malicious attacks. It monitors and filters all incoming and outgoing HTTP traffic to web applications in order to detect and prevent various types of attacks, including SQL injection, cross-site scripting (XSS), cross-site request forgery (CSRF), etc.
[0004] Both solutions are designed to ensure the security of the entire business system, but the existing technology is that the two solutions perform their respective functions without regular linkage and interaction. In response to the client's continuous malicious scanning and sniffing of web system vulnerabilities, openresty will analyze the request method, request body, etc., identify its malicious actions and intercept them, and return abnormal status codes such as 403. However, due to its working characteristics, iptables cannot accurately identify malicious traffic and continues to release it, making it very likely to bypass the waf rules after multiple attempts and carry out the next step of damage, thus creating a defense gap. Summary of the invention
[0005] In order to overcome the shortcomings of the prior art and solve the technical problems of no linkage between openresty and iptables and poor defense performance in the existing network system defense, the present invention provides a system based on Openresty to identify abnormal traffic and link iptables to perform IP blocking.
[0006] The present invention is achieved through the following technical solutions.
[0007] The present invention provides a system for identifying abnormal traffic based on Openresty and linking iptables to perform IP blocking, including an openresty waf module, a waf rule module, an abnormal traffic recording module, a linkage blocking determination module, a recording and notification module, and an audit module;
[0008] The openresty waf module is used to add relevant waf configurations to the openresty configuration file; the waf rule module is used for rule matching and intercepting malicious requests; the abnormal traffic recording module is used to record and store security events and attack information through redis when malicious requests trigger waf rules; the linkage blocking judgment module is used to determine whether to trigger the linkage blocking judgment rule and perform blocking operations; the recording and notification module is used to record logs for audit and notify the network and security managers; the audit module is used to manually audit and identify intercepted requests;
[0009] The linkage blocking judgment module performs secondary filtering on the basis of WAF interception, and blocks the IP that meets the linkage blocking characteristics; the linkage blocking judgment module performs DROP operation in the input or prerouting chain through the iptables configuration interface according to the linkage rules; the linkage rules are judged according to the priority of basic rules; the basic rules include request parameter abnormalities containing sql or exe or sh keywords, request time period between 00:00-6:00, request IP address matches a foreign IP address in the address library, the number of requests is higher than 5 times per second for the same request, and the request client type matches a robot or spider or curl abnormal browser client; the priority of the basic rules is that when one or more basic rules are met at the same time, the IP address source is the highest, and if the IP address match is abnormal, linkage blocking is performed, and other rules are no longer judged; if there is no abnormality in the IP address match, adaptation is performed in sequence according to the request client, request method, request parameter, request number, and request time period, and IP blocking operation is performed if there is a hit.
[0010] Furthermore, the waf configuration includes specifying a path of a waf lua script, specifying an initialization script of the waf, and specifying an access control script of the waf.
[0011] Furthermore, the rules of the waf rule module include SQL injection detection rules, XSS detection rules, command injection detection rules, file inclusion detection rules and HTTP protocol violation detection rules; the SQL injection detection rules identify SQL injection attacks by checking whether the request parameters contain SQL statement keywords or semicolons or quotation marks; the XSS detection rules identify XSS injection attacks by checking whether the request parameters contain HTML or JavaScript codes or angle brackets or quotation marks; the command injection detection rules identify command injection attacks by checking whether the request parameters contain system command execution keywords or characters; the file inclusion detection rules identify file inclusion attacks by checking whether the request parameters contain file path related keywords or characters; the HTTP protocol violation detection rules identify malicious requests or protocol violations by checking whether the request header, request method, and HTTP version comply with the specifications.
[0012] Furthermore, the linkage rules also include WAF identifying SQL injection detection, directly triggering linkage blocking, requesting time period or client IP address anomalies directly blocking its access and listing it in the audit system for identification, and client type anomalies directly triggering linkage blocking rules.
[0013] Further, the method for identifying abnormal traffic based on Openresty and linking iptables to perform IP blocking includes the following steps:
[0014] 1) User traffic enters iptables through the iptables layer 4 firewall for judgment. If it is judged to be normal, it passes through the openresty waf layer 7 firewall. If it is judged to be abnormal and fails, it is recorded in the interception log and audit system for audit;
[0015] 2) User traffic passing through the openresty waf seven-layer firewall enters the openresty waf for judgment. If it is judged to be normal, the business operation starts. If it is judged to be abnormal, it is recorded in the interception log and published to redis, and the blocking judgment module is linked to subscribe to the redis information in real time;
[0016] 3) After the linkage blocking judgment module subscribes to the redis information in real time, it will perform basic rule priority screening according to the IP address, request client, request method, request parameters, number of requests, and request time period. If it is judged to be abnormal, it will call the iptables configuration interface to perform network layer blocking operations and record them in the interception log and audit system for auditing.
[0017] The beneficial effects achieved by the present invention are as follows: the present invention proposes a system and method for identifying abnormal traffic based on openresty and linking iptables to perform IP blocking. Openresty is linked with iptables. Openresty uses lua scripts to write customized protection rules to perform in-depth detection and protection on HTTP requests. Iptables is a firewall tool on the Linux system that can filter and process data packets at the IP level. By linking the two, comprehensive protection capabilities can be achieved, which can protect against both application layer attacks and network layer attacks, avoiding defense gaps caused by relying solely on one party, and having high processing efficiency. By sharing the traffic processing tasks to openresty and iptables, the overall traffic processing capability and response speed can be improved. At the same time, it has good compatibility and scalability, and can be deployed and expanded in the existing network environment without the need for large-scale system transformation and migration. Both openresty and iptables provide flexible configuration options and management interfaces, which can be customized and managed according to actual needs. By linking the two, protection rules, filtering rules, etc. can be adjusted according to specific circumstances to flexibly respond to different attacks and traffic situations. Through the combination of open source components, powerful four-layer and seven-layer firewall functions are realized, reducing server load and bandwidth consumption, thereby saving server resources and bandwidth costs as well as security equipment costs. The combination of the two also simplifies system architecture and configuration management, reduces maintenance workload and labor costs, and implements more sophisticated request filtering and forwarding strategies, improves the system's response speed and processing capabilities, thereby improving user experience and satisfaction, and increasing user retention and conversion rates.
[0018] Compared with the prior art, the present invention has the advantages of high comprehensive protection capability, combination of openresty and iptables linkage, improved traffic processing efficiency, and reduced manual maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0020] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.
[0021] like Figure 1 As shown, the present invention provides a system for identifying abnormal traffic based on Openresty and linking iptables to perform IP blocking, including an openresty waf module, a waf rule module, an abnormal traffic recording module, a linkage blocking determination module, a recording and notification module, and an audit module.
[0022] The openresty waf module is used to add relevant waf configuration in the openresty configuration file, and the waf configuration includes specifying the path of the waf lua script, specifying the waf initialization script and specifying the waf access control script.
[0023] The waf rule module is used for rule matching and intercepting malicious requests, mainly by using regular expressions, keyword matching and other methods to achieve rule matching. The rules of the waf rule module include SQL injection detection rules, XSS detection rules, command injection detection rules, file inclusion detection rules and HTTP protocol violation detection rules. The SQL injection detection rule identifies SQL injection attacks by checking whether the request parameters contain SQL statement keywords or semicolons or quotation marks. The SQL statement keywords include SELECT, INSERT and UPDATE. The XSS detection rule identifies XSS injection attacks by checking whether the request parameters contain HTML or JavaScript code or angle brackets or quotation marks. The command injection detection rule identifies command injection attacks by checking whether the request parameters contain system command execution keywords or characters. The system command execution keywords include;, | and &&. The file inclusion detection rule identifies file inclusion attacks by checking whether the request parameters contain file path related keywords or characters. The file path related keywords include.. / , %2e%2e / . The HTTP protocol violation detection rule identifies malicious requests or protocol violations by checking whether the request header, request method, and HTTP version meet the specifications.
[0024] The abnormal traffic recording module is used to record and store security events and attack information through redis when malicious requests trigger WAF rules.
[0025] The linkage blocking judgment module performs secondary filtering based on the WAF interception, and blocks the IP that meets the linkage blocking characteristics; the linkage blocking judgment module is used to determine whether the linkage blocking judgment rule is triggered and perform blocking operations, mainly by subscribing to redis related messages, and can obtain log data from redis, and perform analysis, monitoring or auditing in the background. The linkage blocking judgment module performs a DROP operation in the input or prerouting chain through the iptables configuration interface according to the linkage rules. The linkage rules are judged according to the priority of the basic rules; the basic rules include the request parameter abnormalities containing sql or exe or sh keywords, the request time period between 00:00-6:00, the request IP address matches a foreign IP address in the address library, the number of requests is higher than 5 times per second for the same request, and the request client type matches a robot or spider or curl abnormal browser client; the priority of the basic rules is that when one or more basic rules are met at the same time, the IP address source is the highest, if the IP address match is abnormal, the linkage blocking is performed, and other rules are no longer judged; if there is no abnormality in the IP address match, it is adapted in sequence according to the request client, request method, request parameter, number of requests, and request time period, and the IP blocking operation is performed if there is a hit. The linkage rules also include WAF identifying SQL injection detection, directly triggering linkage blocking, because normal business request parameters cannot carry such parameters; request time periods such as midnight time periods, holidays or client IP address abnormalities directly block their access and include them in the audit system for identification; client type abnormalities such as illegal crawlers or maliciously constructed clients directly trigger linkage blocking rules. Through the linkage between openresty and iptables, openresty uses lua scripts to write custom protection rules to perform in-depth detection and protection on HTTP requests; iptables is a firewall tool on the Linux system that can filter and process packets at the IP level. By linking the two, comprehensive protection capabilities can be achieved, which can protect against both application layer attacks and network layer attacks, avoiding defense gaps caused by relying solely on one party, and has high processing efficiency. By sharing the traffic processing tasks to openresty and iptables, the overall traffic processing capabilities and response speed can be improved. At the same time, it has good compatibility and scalability, and can be deployed and expanded in the existing network environment without the need for large-scale system transformation and migration; both openresty and iptables provide flexible configuration options and management interfaces, which can be customized and managed according to actual needs. By linking the two, protection rules, filtering rules, etc. can be adjusted according to specific circumstances to flexibly respond to different attacks and traffic situations.
[0026] The recording and notification module is used to record logs for audit and notify network and security managers via email, text messages, etc.
[0027] The audit module is used to manually audit and identify intercepted requests to ensure that normal traffic is not intercepted by mistake. At the same time, the audit module can also add intercepted requests, IP addresses, etc. to the whitelist or blacklist them in an emergency.
[0028] The method for identifying abnormal traffic based on Openresty and linking iptables to block IP includes the following steps:
[0029] 1) User traffic enters iptables through the iptables layer 4 firewall for judgment. If it is judged to be normal, it passes through the openresty waf layer 7 firewall. If it is judged to be abnormal and fails, it is recorded in the interception log and audit system for audit;
[0030] 2) User traffic passing through the openresty waf seven-layer firewall enters the openresty waf for judgment. If it is judged to be normal, the business operation starts. If it is judged to be abnormal, it is recorded in the interception log and published to redis, and the blocking judgment module is linked to subscribe to the redis information in real time;
[0031] 3) After the linkage blocking judgment module subscribes to the redis information in real time, it will perform basic rule priority screening according to the IP address, request client, request method, request parameters, number of requests, and request time period. If it is judged to be abnormal, it will call the iptables configuration interface to perform network layer blocking operations and record them in the interception log and audit system for auditing.
[0032] Through the combination of open source components, powerful four-layer and seven-layer firewall functions are achieved, reducing server load and bandwidth consumption, thereby saving server resources and bandwidth costs as well as security equipment costs. The combination of the two also simplifies system architecture and configuration management, reduces maintenance workload and labor costs, and implements more sophisticated request filtering and forwarding strategies, improving the system's response speed and processing capabilities, thereby improving user experience and satisfaction, and increasing user retention and conversion rates.
[0033] The above describes in detail the implementation modes of the present invention in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes. The implementation modes can still be changed within the knowledge scope of ordinary technicians in the field. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A system based on Openresty to identify abnormal traffic and link iptables to block IP, characterized by: Including openresty waf module, waf rule module, abnormal traffic recording module, linkage blocking judgment module, recording and notification module and audit module; The openresty waf module is used to add relevant waf configurations to the openresty configuration file; the waf rule module is used for rule matching and intercepting malicious requests; the abnormal traffic recording module is used to record and store security events and attack information through redis when malicious requests trigger waf rules; the linkage blocking judgment module is used to determine whether to trigger the linkage blocking judgment rule and perform blocking operations; the recording and notification module is used to record logs for audit and notify the network and security managers; the audit module is used to manually audit and identify intercepted requests; The linkage blocking judgment module performs secondary filtering on the basis of WAF interception, and blocks IPs that meet the linkage blocking characteristics; the linkage blocking judgment module performs DROP operation in the input or prerouting chain through the iptables configuration interface according to the linkage rules; the linkage rules are judged according to the priority of basic rules; the basic rules include request parameter abnormalities containing sql or exe or sh keywords, request time period between 00:00-6:00, request IP address matches a foreign IP address in the address library, the number of requests is higher than 5 times per second for the same request, and the request client type matches a robot or spider or curl abnormal browser client; the priority of the basic rules is that when one or more basic rules are met at the same time, the IP address source is the highest, and if the IP address match is abnormal, linkage blocking is performed, and other rules are no longer judged; if there is no abnormality in the IP address match, adaptation is performed in sequence according to the request client, request method, request parameter, request number, and request time period, and IP blocking operation is performed if there is a hit.
2. According to claim 1, the system for identifying abnormal traffic based on Openresty and linking iptables to perform IP blocking is characterized in that: The waf configuration includes specifying the path of the waf lua script, specifying the waf initialization script and specifying the waf access control script.
3. The system for identifying abnormal traffic based on Openresty and linking iptables to perform IP blocking according to claim 2 is characterized in that: The rules of the waf rule module include SQL injection detection rules, XSS detection rules, command injection detection rules, file inclusion detection rules and HTTP protocol violation detection rules; the SQL injection detection rules identify SQL injection attacks by checking whether the request parameters contain SQL statement keywords or semicolons or quotation marks; the XSS detection rules identify XSS injection attacks by checking whether the request parameters contain HTML or JavaScript codes or angle brackets or quotation marks; The command injection detection rule identifies command injection attacks by checking whether the request parameters contain system command execution keywords or characters; The file inclusion detection rule identifies file inclusion attacks by checking whether the request parameters contain file path related keywords or characters; the HTTP protocol violation detection rule identifies malicious requests or protocol violations by checking whether the request header, request method, and HTTP version comply with the specifications.
4. The system for identifying abnormal traffic based on Openresty and linking iptables to perform IP blocking according to claim 3 is characterized in that: The linkage rules also include WAF identifying SQL injection detection, directly triggering linkage blocking, requesting time period or client IP address anomalies, directly blocking access and listing it in the audit system for identification, and client type anomalies directly triggering linkage blocking rules.
5. A method for identifying abnormal traffic based on Openresty and linking iptables to block IP, characterized by: The following steps are involved: 1) User traffic enters iptables through the iptables layer 4 firewall for judgment. If it is judged to be normal, it passes through the openresty waf layer 7 firewall. If it is judged to be abnormal and fails, it is recorded in the interception log and audit system for audit; 2) User traffic passing through the openresty waf seven-layer firewall enters the openresty waf for judgment. If it is judged to be normal, the business operation starts. If it is judged to be abnormal and not passed, it is recorded in the interception log and audit system and published to redis. The blocking judgment module is linked to subscribe to the redis information in real time; 3) After the linkage blocking judgment module subscribes to the redis information in real time, it will perform basic rule priority screening according to the IP address, request client, request method, request parameters, number of requests, and request time period. If it is judged to be abnormal, it will call the iptables configuration interface to perform network layer blocking operations and record them in the interception log and audit system for auditing.