Webpage application traffic auditing method and device, storage medium and program product

Abstract

The embodiment of the invention discloses a webpage application traffic auditing method and device, a storage medium and a program product. The method comprises the following steps: acquiring and analyzing a plurality of access rule templates of a website, and constructing an access path matching handle and a domain name matching handle based on analyzed template data; extracting and analyzing a network protocol message from the webpage application flow, and obtaining an actual access path, an actual domain name and an actual request method; based on the access path matching handle, the domain name matching handle, the actual access path and the actual domain name, screening out at least one candidate template from the plurality of access rule templates; based on an actual request method, determining whether an available template exists in the at least one candidate template; and if yes, extracting audit data from the network protocol message based on the available template, and generating an audit log. According to the method, personalized auditing of the webpage application traffic can be realized by combining the access rule template and the matching handle, and the auditing data capturing efficiency and accuracy are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of network security technologies, and in particular, to a method, device, storage medium, and program product for auditing web application traffic. Background Art

[0002] In the context of the increasingly severe current network security situation, the HTTPS protocol has become a key means of protecting data privacy due to its encryption transmission mechanism. However, this encryption feature also makes it difficult for traditional auditing methods to directly parse and audit its content. In web application traffic auditing, different websites and services often have unique business logics and data interaction patterns, but traditional HTTPS auditing methods are difficult to accurately capture the data in encrypted traffic, and the generated audit logs cannot meet the diverse auditing requirements of different websites and services, thus affecting the accuracy and efficiency of auditing. Summary of the Invention

[0003] In view of this, embodiments of the present disclosure provide a method, device, storage medium, and program product for auditing web application traffic, which can achieve personalized auditing of web application traffic by combining access rule templates and matching handles, and improve the efficiency and accuracy of capturing audit data.

[0004] In a first aspect, embodiments of the present disclosure provide a method for auditing web application traffic, adopting the following technical solution: Obtain and parse multiple access rule templates of a website, and based on the parsed template data, construct an access path matching handle and a domain name matching handle; Extract network protocol packets from web application traffic, parse the network protocol packets to obtain the actual access path, actual domain name, and actual request method; Based on the access path matching handle, the domain name matching handle, the actual access path, and the actual domain name, filter out at least one candidate template from the multiple access rule templates; Based on the actual request method, determine whether there is an available template among the at least one candidate template; If there is, based on the available template, extract audit data from the network protocol packets and generate an audit log based on the audit data.

[0005] Optionally, the constructing an access path matching handle and a domain name matching handle based on the parsed template data includes: Store the template data in an array, and use the index of the array as the identifier of the corresponding access rule template; Traverse each array storing template data in sequence, and obtain the access path and domain name of each access rule template; Convert each access path and domain name into an access path matching pattern string and a domain name matching pattern string respectively; Compile the access path matching pattern string and the domain name matching pattern string respectively to generate an access path matching handle and a domain name matching handle.

[0006] Optionally, screening at least one candidate template from the multiple access rule templates based on the access path matching handle, the domain name matching handle, the actual access path, and the actual domain name includes: Match the actual access path with the access path in each access rule template through the access path matching handle; If an identifier returned by the access path matching handle is received, screen at least one matching template from the multiple access rule templates based on the identifier returned by the access path matching handle; Match the actual domain name with the domain name in each matching template through the domain name matching handle; If an identifier returned by the domain name matching handle is received, confirm at least one candidate template from the screened access rule templates based on the identifier returned by the domain name matching handle.

[0007] Optionally, determining whether there is an available template in the at least one candidate template based on the actual request method includes: Screen a target array from multiple arrays based on the subscript of each array and the identifier of the candidate template; Extract the request method of the candidate template from the target array; When the actual request method is the same as the request method of the candidate template, determine the candidate template as an available template.

[0008] Optionally, extracting audit data from the network protocol packet based on the available template includes: Extract keyword content from the network protocol packet based on the identifier of the available template and a preset field linked list set; Determine a target log type based on the identifier of the available template; Screen a target log template from multiple preset log templates based on the target log type; Extract audit data from the keyword content based on the target log template.

[0009] Optionally, the web application traffic audit method further includes: Traverse each array storing template data in sequence to obtain the keyword objects of each access rule template, where the keyword objects include at least one of the content format corresponding to the keyword, the position where the keyword is located, and the keyword path; Create multiple empty linked lists, and based on the position where the keyword is located, store the content format corresponding to the keyword, the keyword path, and the identifier of the access rule template into the corresponding empty linked lists to obtain multiple field linked lists, and the multiple field linked lists form a field linked list set.

[0010] Optionally, the extracting the keyword content from the network protocol packet based on the identifier of the available template and the preset field linked list set includes: Based on the identifier of the available template, extract the content format corresponding to the target keyword and the target keyword path from each field linked list; Based on the content format corresponding to the target keyword and the target keyword path, extract the keyword content from the network protocol packet.

[0011] In a second aspect, an embodiment of the present disclosure further provides a web application traffic auditing system, which adopts the following technical solutions: A template parsing module, configured to obtain and parse multiple access rule templates of a website, and based on the parsed template data, construct an access path matching handle and a domain name matching handle; A packet parsing module, configured to extract a network protocol packet from web application traffic, parse the network protocol packet, and obtain an actual access path, an actual domain name, and an actual request method; A first screening module, configured to screen out at least one candidate template from the multiple access rule templates based on the access path matching handle, the domain name matching handle, the actual access path, and the actual domain name; A second screening module, configured to determine whether there is an available template in the at least one candidate template based on the actual request method; A log generation module, configured to, if so, extract audit data from the network protocol packet based on the available template, and generate an audit log based on the audit data.

[0012] In a third aspect, an embodiment of the present disclosure further provides a computer device, which adopts the following technical solutions: The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the web application traffic auditing method described in any one of the above.

[0013] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium that stores computer instructions for causing a computer to execute the web application traffic auditing method described in any one of the above.

[0014] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, the steps of the method described in any one of the above are implemented.

[0015] The web application traffic auditing method provided by the embodiments of the present disclosure can accurately match the actual access path and actual domain name with the rule templates by obtaining and parsing multiple access rule templates of a website and constructing access path matching handles and domain name matching handles, so as to avoid the fuzzy matching problems that may occur in traditional auditing methods and make the auditing process more accurate. Moreover, compared with checking all access rules one by one, this screening method based on matching handles greatly reduces the time and computing resources required for auditing. After screening out candidate templates, it is further determined whether there are available templates based on the actual request method, enabling the auditing to more deeply consider the business scenario. Since the available templates are constructed according to the actual access rules of the website, key information related to auditing can be accurately extracted, avoiding the situation of extracting irrelevant data. An audit log can be generated based on the extracted audit data, which can be used for subsequent security analysis, compliance checking, fault troubleshooting, etc., providing comprehensive audit clues for security managers.

[0016] The above description is only an overview of the technical solutions of the present disclosure. In order to understand the technical means of the present disclosure more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features, and advantages of the present disclosure more obvious and understandable, the following preferred embodiments are specifically given and described in detail in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings required for the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic flowchart of the web application traffic auditing method provided by the embodiments of the present disclosure; Figure 2 Schematic flowchart of the matching handle construction method provided by an embodiment of the present disclosure; Figure 3 Schematic flowchart of the candidate template screening method provided by an embodiment of the present disclosure; Figure 4 Schematic flowchart of the audit data extraction method provided by an embodiment of the present disclosure; Figure 5 Principle block diagram of the web application traffic audit system provided by an embodiment of the present disclosure; Figure 6 Schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure. Detailed implementation manners

[0019] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0020] It should be clear that the following uses specific specific examples to illustrate the implementation manners of the present disclosure, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts belong to the scope of protection of the present disclosure.

[0021] It should be noted that the following describes various aspects of the embodiments within the scope of the appended claims. It should be obvious that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on the present disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, this device can be implemented and this method can be practiced using other structures and / or functions in addition to one or more of the aspects described herein.

[0022] It should also be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present disclosure in a schematic manner, and only show the components related to the present disclosure in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The type, quantity and ratio of each component in its actual implementation can be an arbitrary change, and the component layout type may also be more complex.

[0023] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the aspects described can be practiced without these specific details.

[0024] Referring to Figure 1 , the present disclosure provides a method for auditing web application traffic, including the following steps: S1: Obtain and parse multiple access rule templates of a website, and based on the parsed template data, construct an access path matching handle and a domain name matching handle; S2: Extract network protocol packets from the web application traffic, parse the network protocol packets, and obtain the actual access path, actual domain name, and actual request method; S3: Based on the access path matching handle, domain name matching handle, actual access path, and actual domain name, screen out at least one candidate template from multiple access rule templates; S4: Based on the actual request method, determine whether there is an available template among at least one candidate template; if so, execute S5; if not, execute S6; S5: Based on the available template, extract audit data from the network protocol packets, and generate an audit log based on the audit data; S6: Determine that personalized auditing of the web application traffic is not required.

[0025] The method for auditing web application traffic provided by the present disclosure can accurately match the actual access path and actual domain name with the rule templates by obtaining and parsing multiple access rule templates of the website and constructing an access path matching handle and a domain name matching handle, so as to avoid the fuzzy matching problem that may occur in traditional auditing methods and make the auditing process more accurate. For example, for a complex website structure, different sub-domains and paths may have different access rules, and this method can accurately identify whether each specific access conforms to the corresponding rules, greatly improving the accuracy of auditing. Moreover, compared with checking all access rules one by one, this screening method based on matching handles greatly reduces the time and computing resources required for auditing, especially when dealing with a large amount of web application traffic, it can significantly improve the auditing efficiency and timely discover potential security problems.

[0026] After screening out the candidate templates, further determine whether there is an available template based on the actual request method. Different request methods (such as GET, POST, PUT, DELETE, etc.) may correspond to different business logics and security requirements. This screening mechanism combined with the request method enables auditing to more deeply consider the business scenarios, and only further audit the traffic that conforms to the rules of specific request methods, avoiding unnecessary audit operations and improving the pertinence of auditing.

[0027] Since the available templates are constructed according to the actual access rules of the website, they can accurately extract key information related to auditing, avoiding the extraction of irrelevant data. This makes the audit data more effective, providing a more valuable basis for subsequent analysis and decision-making, ensuring the pertinence of web application auditing, and achieving personalized auditing of web application traffic. Based on the extracted audit data, audit logs are generated to ensure the integrity of audit information. The audit logs record detailed information about web application traffic, including the actual access path, actual domain name, actual request method, and relevant audit data, which can be used for subsequent security analysis, compliance checking, and troubleshooting, providing comprehensive audit clues for security managers.

[0028] In S1, before conducting web application auditing, according to specific audit requirements, access rule templates are customized for the corresponding websites. Different usage scenarios of the website are fully considered, such as the operation permission differences of different user roles (such as ordinary visitors, registered users, administrators, etc.), diverse business scenarios (such as product browsing, transaction payment, information management, etc.), and special access requirements in different time periods (such as weekdays and holidays, day and night, etc.), and corresponding access rule templates are constructed. These constructed access rule templates are stored in the database, and the access rule templates are read from the database and parsed in detail to obtain the template data contained in the access rule templates for the subsequent development of audit work.

[0029] Among them, the template data includes at least one of the template name, protocol name, cached data type, log type, request method, domain name, access path, and keyword object. Among them, the log type includes at least one of email logs, forum logs, network disk logs, and search engine logs; the domain name can be selected as the host name (Host); the access path can be selected as the uniform resource locator (URL); the keyword object includes at least one of the keyword corresponding content format, the location where the keyword is located, and the keyword path. The keyword corresponding content format includes at least one of json, urlparam, xml, html, mime, line_base, and regex; the location where the keyword is located includes at least one of the access path, request header, request body, response header, and response body.

[0030] Refer to Figure 2 Referring to the flowchart of the method for constructing a matching handle shown, "Based on the parsed template data, construct an access path matching handle and a domain name matching handle" includes the following steps: S11: Store the template data in an array, and use the index of the array as the identifier of the corresponding access rule template; S12: Traverse each array storing template data in sequence, and obtain the access path and domain name of each access rule template; S13: Convert each access path and domain name into an access path matching pattern string and a domain name matching pattern string respectively; S14: Compile the access path matching pattern string and the domain name matching pattern string respectively to generate an access path matching handle and a domain name matching handle.

[0031] In S11, when each access rule template is parsed, an array is created, and according to the creation order, the index of the array increases naturally. The parsed template data is stored in this array, and the index of this array serves as the unique identifier of this access rule template. By this method, not only can the storage of template data be completed, but each access rule template can also be uniquely identified.

[0032] In S12 - S14, traverse each array in sequence, and extract the access path and domain name from the template data stored in the array.

[0033] For the access path, focus on its query string part. The parameter values in the query string often change dynamically. To make the generated pattern string able to match different parameter values, use the replacement function of regular expressions to replace the parameter values in the query string with a pattern that can represent any alphanumeric character sequence. For example, use \w+ to replace the specific parameter values. Check the possible dynamic parts in the access path, such as consecutive numbers. Similarly, use regular expression replacement to replace these consecutive number parts with a pattern that can match any consecutive numbers, that is, \d+. After these two steps of processing, the access path is converted into an access path matching pattern string with a certain generality.

[0034] Although the domain name is relatively fixed, to consider various possibilities of sub - domains, perform wildcard processing on the domain name. Through regular expression replacement, replace the sub - domain part of the domain name with a pattern that can match any sub - domain, allowing different sub - domain situations to be matched, so as to obtain the domain name matching pattern string.

[0035] Use the regular expression compilation function provided by the hypscan dynamic library (such as the hs_compile() function in the Libhs library), and pass the access path matching pattern string as a parameter to this function. During the compilation process, the pattern string will be parsed and optimized to generate a regular object, which is the access path matching handle. This handle can be used for subsequent matching operations on the actual access path, and due to the compilation and optimization, the matching efficiency will be higher. Similarly, use the regular expression compilation function, pass the domain name matching pattern string as a parameter, and after compilation, generate a new regular object, that is, the domain name matching handle, which can be used for matching operations on the actual domain name to improve the efficiency of domain name matching. And this method uses the Hyperscan algorithm to complete the compilation of the pattern string and generate the corresponding matching handle. The Hyperscan compiler can compile a large pattern database into a small memory footprint, which is suitable for running in the processor cache and reduces external memory access. This not only improves the speed of pattern matching but also optimizes the overall system performance, especially suitable for scenarios that need to process high-throughput data.

[0036] In S2, configure Nginx to act as an HTTPS transparent proxy to intercept the HTTPS traffic of web applications. During the configuration process, load the certificates corresponding to each web application. Nginx uses the loaded certificates to decrypt the intercepted web application traffic and convert the encrypted HTTPS traffic into plaintext HTTP traffic to prepare for subsequent parsing work. Subsequently, forward the decrypted HTTP traffic to the HTTP parsing plugin. The HTTP parsing plugin extracts the network protocol packets (HTTP packets) from the decrypted HTTP traffic. HTTP packets include request packets and response packets. The HTTP parsing plugin focuses on parsing the headers of the request packets. The request packet headers contain many important information, such as the request method, the domain name of the request, and the access path, etc. By parsing these header fields, the required actual access path, actual domain name, and actual request method can be extracted.

[0037] In S3, refer to Figure 3 the flow schematic diagram of the candidate template screening method shown, "select at least one candidate template from multiple access rule templates based on the access path matching handle, domain name matching handle, actual access path, and actual domain name" includes the following steps: S31: Use the access path matching handle to match the actual access path with the access paths in each access rule template; S32: If the identifier returned by the access path matching handle is received, based on the identifier returned by the access path matching handle, select at least one matching template from multiple access rule templates; S33: Match the actual domain name with the domain names in each matching template through the domain name matching handle; S34: If the identifier returned by the domain name matching handle is received, based on the identifier returned by the domain name matching handle, confirm at least one candidate template from the filtered access rule templates.

[0038] In the above steps, call the access path matching handle, which will match the actual access path with the access paths in each access rule template one by one. If the actual access path matches the access path in any access rule template successfully, the access path matching handle will return the identifier of that access rule template. Based on the identifier returned by the access path matching handle, matching templates can be filtered out from all access rule templates, and most of the unavailable access rule templates can be excluded.

[0039] Further call the domain name matching handle, which will match the actual domain name with the domain names in each matching template one by one. If the actual domain name matches the domain name in the template successfully, the domain name matching handle will return the corresponding identifier. Based on the identifier returned by the domain name matching handle, candidate templates can be further confirmed from the filtered matching templates, and the filtering range can be further narrowed down.

[0040] The access path usually has a high degree of discrimination. First, perform access path matching, and then perform domain name matching. This hierarchical filtering method can gradually refine the filtering results. Access path matching, as the first layer of filtering, can quickly exclude a large number of irrelevant templates. Domain name matching, as the second layer of filtering, can further confirm candidate templates within a smaller range, ensuring the accuracy of the filtering results. Through this hierarchical filtering, it is also possible to avoid full-scale matching of all templates, reduce unnecessary consumption of computing resources, and improve the overall performance and filtering efficiency of the system.

[0041] If the identifier returned by the access path matching handle is not received, or the identifier returned by the domain name matching handle is not received, then use the conventional audit method of the HTTP protocol (such as a Web security scanning tool or an httpx audit tool) to capture audit data from the network protocol packets and generate audit logs. For example, parse the request headers to identify fields such as Content-Type, User-Agent, and Cookie; parse the request body, and according to the different Content-Type (such as application / json, application / x-www-form-urlencoded, etc.), use the corresponding method to parse the data in the request body, record the parsing details, and generate http audit logs.

[0042] In S4, based on the subscript of each array and the identifier of the candidate template, the target array is filtered out from multiple arrays. The array with the same subscript as the identifier of the candidate template is determined as the target array, and then the request method of the candidate template is extracted from the target array. The actual request method parsed from the request message is compared with the request method of the candidate template; if they are the same, the candidate template is determined as an available template, indicating that the current web application traffic belongs to the scope of user personalized auditing, and subsequent processing can be carried out according to the personalized auditing rules; if they are different, it means that the current web application traffic is not the object of concern in user personalized auditing. At this time, the conventional auditing method of the HTTP protocol is used to capture the audit data from the network protocol message and generate the corresponding audit log. In the entire filtering process, the access rule template acts as a filter. On the one hand, it can efficiently identify the part that truly needs to be subject to personalized auditing from numerous web application traffic, and accurately exclude the traffic that does not meet the requirements of personalized auditing; on the other hand, the available template applicable to the web application traffic can also be determined synchronously during the filtering process. Through this filtering method, the pertinence of the auditing work can be greatly improved, enabling the auditing resources to be concentrated on key points, and at the same time significantly enhancing the overall efficiency of the auditing work. In addition, the matching calculation for the access path and domain name is relatively complex, and the matching handle method can efficiently complete the matching process and reduce the calculation complexity. While the matching calculation for the request method is relatively simple, so the direct matching method for the request method can save computing resources.

[0043] In S5, by traversing each array storing template data in sequence, the keyword object of each access rule template is obtained. Multiple empty linked lists are created. For example, according to the known field types, namely the access path field, request header field, response header field, request body field, and response body field, the corresponding empty linked lists are created respectively. Based on the position where the keyword is located, the content format corresponding to the keyword, the keyword path, and the identifier of the access rule template are stored in the corresponding empty linked lists, obtaining multiple field linked lists, and the multiple field linked lists form a field linked list set. Among them, the field linked lists include the access path field linked list, request header field linked list, response header field linked list, request body field linked list, and response body field linked list. The access path field linked list is used to store the keyword extraction rules related to the access path, the request header field linked list is used to store the keyword extraction rules related to the request header, the response header field linked list is used to store the keyword extraction rules related to the request body, the request body field linked list is used to store the keyword extraction rules related to the response header, and the response body field linked list is used to store the keyword extraction rules related to the response body.

[0044] Refer to Figure 4 Referring to the flowchart of the audit data extraction method shown, "extracting audit data from the network protocol message based on the available template" includes the following steps: S51: Extract and obtain keyword content from network protocol packets based on the identification of available templates and a preset set of field linked lists; S52: Determine the target log type based on the identification of available templates; S53: Filter the target log template from multiple preset log templates based on the target log type; S54: Extract audit data from the keyword content based on the target log template.

[0045] In S51, based on the identification of available templates, extract the content format corresponding to the target keyword and the target keyword path from each field linked list. Based on the content format corresponding to the target keyword and the target keyword path, extract the keyword content from the network protocol packets and store the keyword content in the map table in the private area. Specifically, according to the identification of the available template, obtain the rules for extracting the access path keyword from the request packet from the access path field linked list. These rules are the content format and path for extracting the access path keyword from the request packet. Similarly, according to the identification of the available template, obtain the rules for extracting the request header keyword from the request packet from the request header field linked list. These rules are the content format and path for extracting the request header keyword from the request packet. Obtain the rules for extracting the response header keyword from the response packet from the response header field linked list. These rules are the content format and path for extracting the response header keyword from the response packet. Obtain the rules for extracting the request body keyword from the request packet from the request body field linked list. These rules are the content format and path for extracting the request body keyword from the request packet. Obtain the rules for extracting the response body keyword from the response packet from the response body field linked list. These rules are the content format and path for extracting the response body keyword from the response packet. With the help of the field linked list and the identification of the available template, the relevant rules for extracting keywords from network protocol packets can be quickly located, and then the keyword extraction of network protocol packets can be quickly completed according to these rules.

[0046] Among them, the target keyword corresponding content format and the target keyword path exist in pairs. For example, when the target keyword corresponding content format is json, the expression of the target keyword path is $. + node name, such as "$.data", "$.data.name"; when the target keyword corresponding content format is urlparam, the expression of the target keyword path is the keyword name, such as "comment"; when the target keyword corresponding content format is xml, the expression of the target keyword path is like " / / string[@name='account'] / text()"; when the target keyword corresponding content format is html, the expression of the target keyword path is like " / / body / div"; when the target keyword corresponding content format is mime, the expression of the target keyword path is like "JSESSIONID", if the field to be retrieved nests other formats, the expression is like: "from" : { "type" : "req_body", "format" : "mime", "path" : { "address" : { "format" : "json", "path" : "id"}}}; when the target keyword corresponding content format is regex, the expression of the target keyword path is a regular expression, such as " / zvideo / \d"; when the target keyword corresponding content format is line_base, the expression of the target keyword path is 'keyword name', such as " 'to' ".

[0047] In S52 - S54, multiple preset log templates include but are not limited to email logs, forum logs, network disk logs, and search engine logs. Each log template has its unique structure and purpose and can record various types of information generated in different scenarios. Based on the identifier of the available template, the log type of the available template is extracted from the corresponding array and is called the target log type. The target log template that matches the target log type is determined from multiple preset log templates, and the keyword content stored in the map table is matched with the target log template, so that the keyword content is screened under the rule framework set by the target log template, and the data that meets the requirements is extracted from the map table. These extracted data are the required audit data.

[0048] Fill the audit data into the target log template to generate audit logs. The system automatically stores the generated audit logs in the format of.csv files, which has good generality and compatibility, facilitating subsequent data processing and analysis. After being stored as.csv files, the audit logs will be imported into the ClickHouse database. ClickHouse is a high-performance columnar database management system with fast data reading and writing capabilities and powerful data analysis functions, capable of efficiently storing and managing large-scale audit log data. After being stored in ClickHouse, operation and maintenance personnel can conveniently query these audit logs on the web interface. Through the intuitive interface display and flexible query functions, operation and maintenance personnel can quickly locate and obtain the required audit information. These audit logs provide strong data support for the in-depth analysis and scientific decision-making of operation and maintenance personnel, helping them promptly discover potential problems in system operation, formulate reasonable optimization strategies, and ensure the stable and efficient operation of the system.

[0049] The present disclosure also provides a specific example: Configure the HTTPS transparent proxy in the deployment mode of the network audit system, and turn on the certificate switch of QQ Mail. Install the root certificate on the terminal connected to the network audit system. After completing these operations, log in to QQ Mail, perform the operation of writing a letter on the mail interface, add the information of the recipient, carbon copy recipient, blind carbon copy recipient, and the body content, and then click the "Send" button. At this time, web application traffic will be generated. By performing template screening operations on this traffic, a template matching the operation of sending an email in QQ Mail can be obtained. Using this template, QQ Mail audit logs can be generated. In the QQ Mail audit logs of the network audit system, detailed information such as the recipient, carbon copy recipient, blind carbon copy recipient, and body content of the letter can be found.

[0050] The following is a content example of an access rule template for the operation of sending an email in QQ Mail: { "Protocol name": "QQ Mail", "Template name": "qq send body", "Cached data type": "content", "Log type": "Mail log", "Domain name": "mail.qq.com", "Request method": "POST", "Access path": " / cgi-bin / compose_send\\?sid=\\w+", "Keyword object": { "Sender": { "Location of keyword": "Request body", "Format of content corresponding to keyword": "urlparam", "Keyword path": "sendmailname" }, "Recipient": { "Location of keyword": "Request body", "Format of content corresponding to keyword": "urlparam", "Keyword path": "to" }, "Carbon copy recipient": { "Location of keyword": "Request body", "Format of content corresponding to keyword": "urlparam", "Keyword path": "cc" }, "Blind carbon copy recipient": { "Location of keyword": "Request body", "Format of content corresponding to keyword": "urlparam", "Keyword path": "bcc" }, "Subject": { "Location of keyword": "Request body", "Format of content corresponding to keyword": "urlparam", "Keyword path": "subject" }, "Body": { "Location of keyword": "Request body", "Format of content corresponding to keyword": "urlparam", "Keyword path": "content__html" }, "Mail ID": { "Location of keyword": "Access path", "Format of content corresponding to keyword": "urlparam", "Keyword path": "mid" } } } In summary, this method combines Nginx proxy with the rules of template configuration fields to decrypt web applications. For the decrypted traffic, keyword information is extracted based on different content formats and paths, and the extracted keyword content is stored and queried using a map table. These operations not only effectively improve the accuracy and effectiveness of audit logs but also ensure their integrity. In addition, this method can also flexibly determine whether to conduct an audit of specific web applications based on the enabled status of web application certificates in certificate management.

[0051] Referring to Figure 5 , the present disclosure provides a web application traffic audit system, including: A template parsing module 101, configured to obtain and parse multiple access rule templates of a website, and based on the parsed template data, construct an access path matching handle and a domain name matching handle; A message parsing module 102, configured to extract a network protocol message from web application traffic, parse the network protocol message, and obtain an actual access path, an actual domain name, and an actual request method; A first screening module 103, configured to screen out at least one candidate template from multiple access rule templates based on the access path matching handle, the domain name matching handle, the actual access path, and the actual domain name; A second screening module 104, configured to determine whether there is an available template in at least one candidate template based on the actual request method; A log generation module 105, configured to, if there is an available template, extract audit data from the network protocol message based on the available template, and generate an audit log based on the audit data.

[0052] The various change methods and specific examples in the above-provided web application traffic audit method are equally applicable to the web application traffic audit system provided by the present disclosure. Through the foregoing detailed description of the web application traffic audit method, those skilled in the art can clearly know the implementation method of the web application traffic audit system. For the sake of simplicity of the specification, it will not be elaborated herein.

[0053] A computer device according to an embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0054] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the computer device executes all or part of the steps of the web application traffic auditing method according to the foregoing embodiments of the present disclosure.

[0055] Those skilled in the art should understand that, in order to solve the technical problem of how to obtain good user experience effects, this embodiment may also include well-known structures such as communication buses, interfaces, etc., and these well-known structures should also be included in the protection scope of the present disclosure.

[0056] As Figure 6 FIG. is a schematic structural diagram of a computer device provided by an embodiment of the present disclosure. It shows a schematic structural diagram of a computer device suitable for implementing the computer device in the embodiments of the present disclosure. Figure 6 The shown computer device is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0057] As Figure 6 As shown, the computer device may include a processor (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0058] Generally, the following devices may be connected to the I / O interface: input devices including, for example, sensors or visual information acquisition devices; output devices including, for example, display screens; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. The communication device may allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or wiredly to exchange data. Although Figure 6 the shown computer device has various devices, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0059] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the web application traffic auditing method according to the embodiments of the present disclosure are performed.

[0060] For a detailed description of this embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, and details will not be repeated here.

[0061] A computer-readable storage medium according to an embodiment of the present disclosure stores non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are run by a processor, all or part of the steps of the web application traffic auditing method according to the foregoing embodiments of the present disclosure are performed.

[0062] The above computer-readable storage medium includes, but is not limited to: optical storage media (such as CD-ROMs and DVDs), magneto-optical storage media (such as MOs), magnetic storage media (such as magnetic tapes or external hard drives), media with built-in rewritable non-volatile memories (such as memory cards), and media with built-in ROMs (such as ROM cartridges).

[0063] For a detailed description of this embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, and details will not be repeated here.

[0064] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above disclosed specific details are only for illustrative and facilitating understanding purposes, and not for limitation. The above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0065] In this disclosure, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any way. Words such as "including", "comprising", "having", etc. are open-ended words, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the word "and / or", and can be used interchangeably with each other, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with each other.

[0066] In addition, as used herein, the "or" used in the listing of items starting with "at least one" indicates a disjunctive listing, so that for example, the listing of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the wording "exemplary" does not mean that the examples described are preferred or better than other examples.

[0067] It should also be noted that in the systems and methods of this disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of this disclosure.

[0068] Various changes, substitutions, and alterations to the technologies described herein can be made without departing from the teachings defined by the appended claims. In addition, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and acts described above. Current or later-developed processes, machines, manufactures, compositions of events, means, methods, or acts that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or acts within their scope.

[0069] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

[0070] The foregoing description has been presented for purposes of illustration and description. In addition, this description is not intended to limit embodiments of the present disclosure to the forms disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some of their variations, modifications, alterations, additions, and subcombinations.

Claims

1. A web application traffic audit method, characterized in that: include: Obtain and parse multiple access rule templates of the website, and build access path matching handles and domain name matching handles based on the parsed template data; Extract network protocol messages from web application traffic, parse the network protocol messages, and obtain actual access paths, actual domain names, and actual request methods; Filtering at least one candidate template from the multiple access rule templates based on the access path matching handle, the domain name matching handle, the actual access path, and the actual domain name; Based on the actual request method, determining whether there is an available template in the at least one candidate template; If so, based on the available template, the audit data is extracted from the network protocol message, and an audit log is generated based on the audit data.

2. The web application traffic audit method according to claim 1, characterized in that: The step of constructing an access path matching handle and a domain name matching handle based on the parsed template data includes: Storing the template data in an array, and using the index of the array as an identifier of a corresponding access rule template; Traverse each array storing template data in turn to obtain the access path and domain name of each access rule template; Convert each access path and domain name into an access path matching pattern string and a domain name matching pattern string respectively; The access path matching pattern string and the domain name matching pattern string are compiled respectively to generate an access path matching handle and a domain name matching handle.

3. The web application traffic audit method according to claim 2, characterized in that: The selecting at least one candidate template from the plurality of access rule templates based on the access path matching handle, the domain name matching handle, the actual access path, and the actual domain name includes: Matching the actual access path with the access path in each access rule template through the access path matching handle; If an identifier returned by the access path matching handle is received, then at least one matching template is selected from the multiple access rule templates based on the identifier returned by the access path matching handle; Matching the actual domain name with the domain name in each matching template through the domain name matching handle; If an identifier returned by the domain name matching handle is received, at least one candidate template is confirmed from the screened access rule templates based on the identifier returned by the domain name matching handle.

4. The web application traffic audit method according to claim 3 is characterized in that: The determining, based on the actual request method, whether there is an available template in the at least one candidate template comprises: Based on the subscript of each array and the identifier of the candidate template, a target array is screened out from the multiple arrays; A request method for extracting the candidate template from the target array; When the actual request method is the same as the request method of the candidate template, the candidate template is determined to be an available template.

5. The web application traffic audit method according to claim 2, characterized in that: The extracting audit data from the network protocol message based on the available template includes: Extracting keyword content from the network protocol message based on the identifier of the available template and the preset field chain list set; Determining a target log type based on the identifier of the available template; Based on the target log type, selecting a target log template from a plurality of preset log templates; Based on the target log template, audit data is extracted from the keyword content.

6. The web application traffic audit method according to claim 5, characterized in that: Also includes: Traversing each array storing template data in turn, obtaining a keyword object of each access rule template, wherein the keyword object includes at least one of a content format corresponding to the keyword, a location of the keyword, and a keyword path; Create multiple empty linked lists, and based on the location of the keyword, store the content format corresponding to the keyword, the keyword path and the identifier of the access rule template in the corresponding empty linked lists to obtain multiple field linked lists, which constitute a field linked list set.

7. The web application traffic audit method according to claim 6, characterized in that: The step of extracting keyword content from the network protocol message based on the identifier of the available template and the preset field chain list set includes: Based on the identifier of the available template, extract the content format corresponding to the target keyword and the target keyword path from each field chain list; Based on the content format corresponding to the target keyword and the target keyword path, keyword content is extracted from the network protocol message.

8. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the web application traffic auditing method described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the web application traffic auditing method described in any one of claims 1-7.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images