Method and apparatus for discovering protocol inconsistency vulnerabilities based on human-machine collaboration

Abstract

This invention provides a method and apparatus for discovering protocol inconsistency vulnerabilities based on human-machine collaboration. The method involves acquiring multiple target middleware to be processed; testing each target middleware using pre-generated test samples to obtain test results; wherein the test samples are extracted and generated based on a pre-stored document analyzer; and analyzing each test result based on a pre-stored difference analysis algorithm to obtain analysis results. If the analysis results include preset differences, then a protocol inconsistency vulnerability is determined to exist among the target middleware. This invention takes a novel human-machine collaborative vulnerability discovery approach, targeting HTTP middleware, and utilizes a document analyzer to extract and generate test samples, along with differential testing methods, to semi-automatically discover protocol inconsistency vulnerabilities caused by semantic differences in various HTTP middleware. This achieves a systematic, scalable, more efficient, and more accurate discovery of protocol inconsistency vulnerabilities.

Description

Technical Field

[0001] This invention relates to the field of network communication security technology, and in particular to a method and apparatus for discovering protocol inconsistency vulnerabilities based on human-machine collaboration. Background Technology

[0002] In existing technologies, HTTP middleware is frequently used between users and the final web application to improve security and performance. This type of architecture is becoming increasingly common in modern cloud-based applications. In operation, a typical HTTP request passes through multiple HTTP middleware processes before reaching its destination. The user sends the request to a front-end server, which then forwards the request to one or more back-end servers.

[0003] If multiple HTTP middlewares exist in the path, and these middlewares have different implementations, then when these different implementations are connected together, semantic and logical inconsistencies may occur. As a result, HTTP communication may be vulnerable to attacks, which could lead to serious security consequences such as cache poisoning, security policy bypassing, or denial-of-service attacks.

[0004] However, systematically discovering protocol inconsistency attacks caused by these semantic differences is no easy task. First, RFC specifications are written in natural human language, lacking standardized structural norms and containing implicit requirements. Extracting this informal, non-standard descriptive language from RFC documents and converting it into formal, standardized, and program-understandable content is difficult. Furthermore, some RFC specifications are very long (for example, RFC 7230, which documents part of the HTTP specification, is 89 pages long), requiring significant manpower to manually extract all HTTP documents, and this manual approach is prone to errors. Second, most logical vulnerabilities caused by semantic gaps cannot be directly translated into any obvious, easily detectable vulnerabilities (such as crashes or memory corruption errors), making them difficult to detect. Previous research has largely relied on ad-hoc manual analysis or analyzed only one type of semantic gap attack. This lacks scalability, fails to provide systematic detection, and is prone to errors and inefficient due to human error. Summary of the Invention

[0005] This invention provides a method and apparatus for mining protocol inconsistency vulnerabilities based on human-machine collaboration, which addresses the shortcomings of existing technologies where human analysis is inaccurate, inefficient, lacks scalability, and cannot systematically detect vulnerabilities. This invention enables systematic, scalable, more efficient, and more accurate mining of protocol inconsistency vulnerabilities.

[0006] This invention provides a method for discovering protocol inconsistency vulnerabilities based on human-machine collaboration, comprising:

[0007] Acquire multiple target middleware to be processed;

[0008] Each of the target middleware is tested using pre-generated test cases to obtain test results; wherein the test cases are extracted and generated based on a pre-stored document analyzer.

[0009] The test results are analyzed based on the pre-stored difference analysis algorithm to obtain the analysis results;

[0010] If the analysis results include preset differences, then it is determined that there is a protocol inconsistency vulnerability among the target middleware.

[0011] According to the present invention, a method for mining protocol inconsistencies based on human-machine collaboration is provided, wherein the pre-stored document analyzer includes:

[0012] A specification requirements finder, which is used to extract specification requirements from RFC documents;

[0013] A text rule converter, which is used to extract and organize natural language sentences in the RFC document through dependency analysis and textual entailment;

[0014] An ABNF rule extractor, which is used to extract valid ABNF rules from the RFC document;

[0015] The ABNF rule adjuster is used to optimize and adjust the valid ABNF rules obtained by the ABNF rule adjuster.

[0016] According to the present invention, a method for discovering protocol inconsistency vulnerabilities based on human-machine collaboration is provided, which extracts and generates test cases based on a pre-stored document analyzer, specifically including:

[0017] Extract grammatical rules, semantic information, valid ABNF rule sets, and sentiment analysis-based specification requirements from RFC documents;

[0018] The specification requirements are transformed into formal expressions through a text rule converter based on dependency analysis and textual implication.

[0019] The test case is generated based on the grammar rules, the semantic information, the valid ABNF rule set, and the formal expression.

[0020] According to the present invention, a method for mining protocol inconsistencies based on human-machine collaboration extracts syntactic rules and semantic information from RFC documents, specifically including:

[0021] Extract natural language sentences from RFC documents using a sentiment-driven, specification-requirement-based finder.

[0022] The natural language sentences are processed using a text rule converter to extract grammatical rules and semantic information from them.

[0023] According to the present invention, a method for mining protocol inconsistency vulnerabilities based on human-machine collaboration extracts valid ABNF rule sets from RFC documents, specifically including:

[0024] Extract valid ABNF rules from RFC documents;

[0025] The effective ABNF rules are optimized to obtain optimized ABNF rules;

[0026] The optimized ABNF rules are used as the effective ABNF rule set.

[0027] According to the present invention, a method for mining protocol inconsistency vulnerabilities based on human-machine collaboration generates test cases based on the syntax rules, the semantic information, the valid ABNF rule set, and the formal expression, specifically including:

[0028] Convert a valid ABNF rule set into an HTTP message;

[0029] Based on the specification requirements, the translator translates the HTTP message into a test case with assertions.

[0030] This invention also provides a protocol inconsistency vulnerability mining device based on human-machine collaboration, comprising:

[0031] The test file acquisition unit is used to acquire multiple target middleware to be processed;

[0032] An attack testing unit is used to test each of the target middlewares using pre-generated test samples to obtain test results; wherein the test samples are extracted and generated based on a pre-stored document analyzer.

[0033] The result analysis unit is used to analyze each of the test results based on a pre-stored difference analysis algorithm to obtain the analysis results;

[0034] The result output unit is used to determine, if the analysis results include preset differences, that there is a protocol inconsistency vulnerability among the target middleware.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the protocol inconsistency vulnerability mining method based on human-machine collaboration as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the protocol inconsistency vulnerability mining method based on human-machine collaboration as described above.

[0037] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the protocol inconsistency vulnerability mining method based on human-computer collaboration as described above.

[0038] This invention provides a method and apparatus for discovering protocol inconsistency vulnerabilities based on human-machine collaboration. The method involves acquiring multiple target middleware to be processed; testing each target middleware using pre-generated test samples to obtain test results; wherein the test samples are extracted and generated based on a pre-stored document analyzer; and analyzing each test result based on a pre-stored difference analysis algorithm to obtain analysis results. If the analysis results include preset differences, then a protocol inconsistency vulnerability is determined to exist among the target middleware. This invention takes a novel human-machine collaborative vulnerability discovery approach, targeting HTTP middleware, and utilizes a document analyzer to extract and generate test samples, along with differential testing methods, to semi-automatically discover protocol inconsistency vulnerabilities caused by semantic differences in various HTTP middleware. This achieves a systematic, scalable, more efficient, and more accurate discovery of protocol inconsistency vulnerabilities. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is one of the flowcharts of the protocol inconsistency vulnerability mining method based on human-machine collaboration provided by the present invention;

[0041] Figure 2 This is the second flowchart of the protocol inconsistency vulnerability mining method based on human-machine collaboration provided by the present invention;

[0042] Figure 3 This is the basic architecture diagram of the specification requirement finder for the protocol inconsistency vulnerability mining method based on human-machine collaboration provided by this invention;

[0043] Figure 4 This is a schematic diagram of the protocol inconsistency vulnerability mining device based on human-machine collaboration provided by the present invention;

[0044] Figure 5This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0045] Figure label:

[0046] 410: Test file acquisition unit; 420: Attack testing unit; 430: Result analysis unit; 440: Result output unit;

[0047] 510: Processor; 520: Communication interface; 530: Memory; 540: Communication bus. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] The following is combined with Figures 1-2 This invention describes a method for detecting protocol inconsistencies based on human-machine collaboration. Figure 1 A flowchart illustrating the dialogue method based on hybrid knowledge management provided by this invention is shown below. Figure 1 As shown, the method includes:

[0050] Step 110: Obtain multiple target middleware to be processed.

[0051] Specifically, middleware refers to software that provides a connection between system software and application software, facilitating communication between various software components, particularly the centralized logic of application software to system software. It is widely used in modern information technology application frameworks such as Web services and service-oriented architectures. This embodiment uses HTTP middleware as an example.

[0052] Over the past few decades, HTTP middleware has seen rapid adoption and deployment, including caching servers, proxy servers, network application firewalls, and content delivery networks (CDNs). They are widely deployed across the internet to improve security and performance. In operation, a typical HTTP request passes through multiple HTTP middleware components before reaching its destination.

[0053] However, if multiple HTTP middleware are present in the path, HTTP communication may be vulnerable to attacks due to inconsistent interpretations of the same HTTP message by these middleware. For example, when a malicious client sends an HTTP message with HTTP request headers that may be interpreted differently by different middleware, the different implementations of these middleware can lead to different interpretations of the same HTTP message. This semantic inconsistency can result in serious security consequences, such as cache poisoning, security policy bypassing, or denial-of-service attacks.

[0054] In recent years, numerous studies have revealed the security problems caused by semantic gap attacks, such as faulty host vulnerabilities, HTTP request smuggling, and denial-of-service attacks resulting from cache poisoning. This demonstrates that such issues have become a serious threat to the internet. Although individual semantic gap attacks have been discovered, previous research has largely relied on ad-hoc manual analysis or analyzed only one type of semantic gap attack. These methods require significant manpower for document analysis and implementation testing, lacking scalability, systematic detection capabilities, and are prone to errors and inefficiency. Therefore, a more systematic, scalable, efficient, and accurate method is needed to explore and discover previously unknown exploit scenarios.

[0055] Step 120: Test each of the target middlewares using pre-generated test cases to obtain test results; wherein the test cases are extracted and generated based on a pre-stored document analyzer.

[0056] Specifically, the document analyzer uses natural language processing techniques to extract two main types of rules: 1) Secification Requirements. These are informal descriptions of the HTTP message processing flow, such as the actions a client, proxy, or server should follow when sending or receiving a specific HTTP request. 2) Extended Backus Normal Form (ABNF) rules. These are standardized formal grammars that describe the structured parsing syntax of the HTTP protocol.

[0057] Test cases are extracted and generated based on a pre-stored document analyzer. Programs with the same functionality but running in different HTTP middleware are used as test targets. The same batch of test cases obtained using the document analyzer are used as input. Each target is tested independently and without interference. The outputs of these HTTP middleware for this batch of inputs are analyzed and compared.

[0058] Step 130: Analyze the test results based on the pre-stored difference analysis algorithm to obtain the analysis results;

[0059] If the analysis results include preset differences, then it is determined that there is a protocol inconsistency vulnerability among the target middleware.

[0060] Specifically, the pre-stored difference analysis algorithm employs a difference fuzzy strategy, utilizing differential testing to discover protocol inconsistency attacks caused by semantic differences. It uses programs with identical functionality running in different HTTP middleware as test targets, and the same batch of test samples obtained using a document analyzer as input. Each target is tested independently and without interference. The outputs of these HTTP middleware programs to this batch of inputs are analyzed and compared. The outputs mainly consist of HTTP logs and HTTP responses generated by the HTTP middleware. Then, by analyzing and comparing the HTTP logs and HTTP responses of each HTTP middleware program, a pair of different HTTP behaviors generated for the same HTTP request is identified. Any behavioral differences present in these analysis results are marked as potential vulnerabilities. Furthermore, this invention can further determine whether a difference conforms to RFC requirements, because this invention has extracted formal rules through a document analyzer, thus enabling rapid location of the root cause of discovered vulnerabilities.

[0061] This invention provides a human-machine collaborative method for discovering protocol inconsistency vulnerabilities. The method involves acquiring multiple target middlewares to be processed; testing each target middleware using pre-generated test samples to obtain test results; wherein the test samples are extracted and generated based on a pre-stored document analyzer; and analyzing each test result based on a pre-stored difference analysis algorithm to obtain analysis results. If the analysis results include preset differences, then a protocol inconsistency vulnerability is determined to exist among the target middlewares. This invention takes a novel human-machine collaborative vulnerability discovery approach, targeting HTTP middleware, and utilizes a document analyzer to extract and generate test samples, along with differential testing methods, to semi-automatically discover protocol inconsistency vulnerabilities caused by semantic differences in various HTTP middlewares. This achieves a systematic, scalable, more efficient, and more accurate method for discovering protocol inconsistency vulnerabilities.

[0062] Based on the above embodiments, in this method, the pre-stored document analyzer includes:

[0063] A specification requirements finder, which is used to extract specification requirements from RFC documents;

[0064] A text rule converter, which is used to extract and organize natural language sentences in the RFC document through dependency analysis and textual entailment;

[0065] An ABNF rule extractor, which is used to extract valid ABNF rules from the RFC document;

[0066] The ABNF rule adjuster is used to optimize and adjust the valid ABNF rules obtained by the ABNF rule adjuster.

[0067] Specifically, RFC (Request For Comments) documents contain almost all important written materials about the Internet. Composed of a series of drafts, starting in 1969, they are a compilation of technical documents about the Internet (initially ARPANET). These documents discuss in detail all aspects of computer networks, focusing on network protocols, processes, programs, concepts, as well as meeting minutes, opinions, and various viewpoints.

[0068] Systematically identifying protocol inconsistency attacks caused by semantic differences in RFC documents is no easy task. First, RFC specifications are written in natural human language, lacking a standardized structure and containing implicit requirements. Extracting this informal, non-standard descriptive language from RFC documents and converting it into formal, standardized, and program-understandable content is difficult. Furthermore, some RFC specifications are very long (for example, RFC 7230, which documents part of the HTTP specification, is 89 pages long), requiring significant manpower to manually extract all HTTP documents, and this manual approach is prone to errors. Second, most logical vulnerabilities caused by semantic gaps cannot be directly translated into any obvious, easily detectable vulnerabilities (such as crashes or memory corruption errors), making them difficult to detect.

[0069] To address this research gap, this invention analyzes and summarizes the root causes of semantic gap attacks resulting from protocol inconsistencies, based on previous work. First, some HTTP middleware server implementations do not comply with RFC requirements. This may be due to developers intentionally relaxing HTTP message detection requirements for compatibility reasons, misunderstanding the RFC requirements, or errors in the code. Second, RFCs define optional requirements, giving developers considerable freedom in their programming. This can lead to inconsistent implementations. For example, RFC documents often use keywords like MAY, SHOULD, or SHOULD NOT to specify these rules. However, developers may implement these rules in different ways. When these different implementations are combined, semantic inconsistencies can arise, making the system vulnerable to attacks.

[0070] Therefore, system solutions for discovering protocol inconsistency attacks caused by such semantic differences rely on a deep understanding of the HTTP protocol specification. This invention explores a method to automatically extract semantic information from RFC documents and guide the generation and mutation of fuzzy test cases, so as to achieve the purpose of human-machine collaborative discovery of protocol inconsistency vulnerabilities.

[0071] Therefore, this invention designs a document analyzer, which consists of four main parts: a specification requirements finder, a text rule converter, an ABNF rule extractor, and an ABNF rule adjuster. The document analyzer uses natural language processing technology to automatically extract rules from RFC documents, primarily extracting specification requirements and Extended Backus Form (ABNF) rules.

[0072] Doctrine requirements are informal descriptions of the HTTP message processing flow, such as the actions that a client, proxy, or server should follow when sending or receiving a specific HTTP request. Sentences containing doctrine requirements tend to use emotionally charged words (e.g., MUST, ought to, not allowed) to emphasize the importance of a constraint, especially those related to security considerations. Therefore, this invention utilizes a doctrine requirement finder based on emotional intensity to extract these sentences with potential doctrine requirements. Subsequently, a text rule converter based on dependency parsing and textual entailment techniques transforms these doctrine requirements into formal, machine-understandable rules.

[0073] Extended Backus Normal Form (ABNF) rules. ABNF is a standardized formal grammar that describes the structured parsable syntax of the HTTP protocol. This invention implements a format feature-based ABNF filter that heuristically extracts ABNF grammar rules from RFCs, including string cleansing, regular expression extraction, case escaping, and prose rule extraction. Based on these extracted semantic and grammatical rules, this invention further generates a large number of test cases suitable for fuzz testing through an extended Backus normal form generator and translator.

[0074] Based on the above embodiments, the method of extracting and generating test samples based on a pre-stored document analyzer specifically includes:

[0075] Extract grammatical rules, semantic information, valid ABNF rule sets, and sentiment analysis-based specification requirements from RFC documents;

[0076] The specification requirements are transformed into formal expressions through a text rule converter based on dependency analysis and textual implication.

[0077] The test case is generated based on the grammar rules, the semantic information, the valid ABNF rule set, and the formal expression.

[0078] Specifically, such as Figure 2 As shown, this invention first combines the main characteristics of the HTTP protocol, utilizes human-computer collaboration, uses a document analyzer to extract RFC documents and generate test cases, and combines differential testing to determine whether the test cases can cause the test target to produce a protocol inconsistency attack due to semantic differences.

[0079] This invention first uses a document analyzer to extract grammatical rules from RFC documents, along with semantic information based on a series of NLP-related technologies. It extracts a valid, canonical ABNF rule set from the RFC through regular expression matching, and simultaneously uses a sentiment analysis-based canonical requirement finder to extract canonical requirements. Finally, it transforms the canonical requirements into formal expressions using a text rule converter based on dependency analysis and textual implication. Using these rules, this invention generates test cases through an ABNF generator and a canonical requirement translator. To trigger as many differences as possible between multiple HTTP servers, this invention also mutates the valid requests generated in the previous steps. Then, this invention uses difference testing to discover semantic gap attacks. It first tests each piece of software independently, then compares logs, requests, and responses to identify combinations of HTTP middleware that respond differently to the same HTTP request, indicating potential vulnerabilities. This invention introduces the concept of HMetrics, which summarizes the observed asymmetries between the behaviors of multiple HTTP implementations. Under different detection models, users can define detection rules based on HMetrics to discover protocol inconsistency attacks caused by semantic differences.

[0080] Based on the above embodiments, the method of extracting syntax rules and semantic information from RFC documents specifically includes:

[0081] Extract natural language sentences from RFC documents using a sentiment-driven, specification-requirement-based finder.

[0082] The natural language sentences are processed using a text rule converter to extract grammatical rules and semantic information from them.

[0083] Specifically, the normative requirement finder based on the intensity of emotion extracts normative requirements from RFC documents. This invention improves upon previous research's traditional regular expression template or keyword-based methods, with an infrastructure such as... Figure 3As shown. Previous research has been inefficient in processing RFC documents. First, RFC documents are described in natural language, not formal language, resulting in complex sentences and flexible expressions. Second, different document authors may have different writing styles, making it difficult to find a universal template that covers most specification requirements. Despite the diversity of writing styles, we found that all specification requirements tend to express a strong emotion, emphasizing the importance of a constraint, especially safety-critical constraints. The more important the specification requirement, the more forceful the description. Therefore, to capture sentences with potential specification requirements, this invention implements a specification requirement finder based on the intensity of emotion using Stanza. This method is better than directly filtering specification requirements using keywords defined in the RFC (such as MUST, SHALL, SHOULD NOT), because some specification requirements still do not use these keywords.

[0084] This text rule converter organizes natural language sentences using dependency parsing and textual entailment. Natural language is flexible; the same semantics can be expressed in multiple forms, including synonym substitution and grammatical variations (such as passive voice). Therefore, converting a specification requirement into a formal expression is not easy. To address this issue, this invention employs dependency parsing and textual entailment techniques to identify grammatical variations, which can clarify the implied meaning within the specification requirement. First, the user needs to provide a message description or a role action as a template assumption in the specification requirement. Field names will automatically adapt to the header names defined in ABNF (i.e., the left-hand side of the ABNF expression). Generally, specification documents will describe common roles in the protocol. Next, dependency parsing is introduced to identify key information in the sentence, such as the target role and HTTP-related fields. Then, this invention can populate a seed template to obtain a specification requirement instance. Finally, textual entailment analysis is performed to categorize the sentence into the seed assumption. It works like an intelligent question-and-answer system, taking a target sentence with a potential specification requirement as a premise and asking whether the sentence implies that assumption (i.e., the user-provided specification requirement seed instance).

[0085] Furthermore, the text rule converter segments a sentence into multiple short sentences based on dependency tree analysis, and then performs text nesting analysis on each of these sentences separately. This makes information inference complete and effective. It can then locate the contextual relationships within the dependency tree and, based on dependency tree analysis, further split a sentence into multiple short sentences. Finally, it performs text nesting analysis on each of these sentences separately, ensuring complete and effective information inference.

[0086] The text rule converter also needs to consider text assignment across sentence structures. Some phrases in the RFC have referential relationships across multiple sentences, such as "this information," "such a request," and "such a URI." The converter needs to identify the implicit references of these phrases to recover the original semantics across multiple sentences. A referential phrase (such as "such a request") can often find its referent sentence in adjacent sentences (such as "a request is..."), and there is no iterative referential relationship. Therefore, the converter also includes a simple forward search algorithm based on keyword fuzzy matching, searching forward (up to 5 sentences) to find the referent sentence. Then, the two sentences are merged into a complex multi-sentence to perform text semantic analysis when the referent sentence is found.

[0087] Based on the above embodiments, the method of extracting valid ABNF rule sets from RFC documents specifically includes:

[0088] Extract valid ABNF rules from RFC documents;

[0089] The effective ABNF rules are optimized to obtain optimized ABNF rules;

[0090] The optimized ABNF rules are used as the effective ABNF rule set.

[0091] Specifically, the ABNF rule extractor extracts valid ABNF rules from RFCs. Most RFCs use ABNF to describe formal specifications, which can describe the syntax of a canonical, parsable structure. This invention collects all relevant RFC documents (RFC 7230-7235) through a data tracking tool, and then implements a format feature-based ABNF filter to heuristically extract ABNF syntax rules from RFCs, including string cleaning, regular expression extraction, case escaping, and prose rule extraction.

[0092] The ABNF rule adjuster optimizes and adjusts the rules obtained by the ABNF rule generator. ABNF rules from different files require adjustments to obtain a final, complete, and error-free set of syntax rules. To optimize ABNF rules, the techniques used in this invention include replacing rule names with case-insensitive rule names, replacing invalid rule definitions with custom rules, renaming rules with the same name in different RFCs, and using the latest RFCs to handle duplicate rule names. The valid ABNF rules are then optimized to obtain optimized ABNF rules, which constitute the valid ABNF rule set.

[0093] Based on the above embodiments, the method, which generates the test cases based on the syntax rules, the semantic information, the valid ABNF rule set, and the formal expression, specifically includes:

[0094] Convert a valid ABNF rule set into an HTTP message;

[0095] Based on the specification requirements, the translator translates the HTTP message into a test case with assertions.

[0096] Specifically, the differential testing phase mainly uses the ABNF rule generator and specification requirement translator to generate test cases, and then uses these cases to perform differential analysis on the target HTTP middleware.

[0097] The ABNF rule generator converts ABNF rule sets into HTTP messages. Based on the type nodes defined in ABNF, the generator parses the ABNF syntax into an ABNF syntax tree. It then locates the target node (e.g., HTTP-message, HTTP-version) as the starting node and recursively traverses the ABNF syntax tree downwards. Leaf nodes (e.g., string characters, numeric characters) are the ending nodes. However, test cases generated directly from the original ABNF syntax tree are often too distorted and easily rejected by the target server. ABNF syntax has variable repetition rules (e.g., n*nRule), theoretically allowing for infinitely deep subtree traversals. However, exhaustive depth traversal of this type of syntax is meaningless. Therefore, we limit the recursive depth of the ABNF syntax tree traversal (e.g., a maximum of 7). Secondly, this invention loads some predefined rules to reduce the generation of invalid strings; these rules can specify certain leaf node values ​​for empirical experimental purposes. For example, the host header can consist of IPv4 addresses. It's not necessary to test all IPv4 addresses; only representative addresses, such as 127.0.0.1 and 8.8.8.8, need to be tested. In this way, basic HTTP requests that fully comply with RFC requirements can be generated and will not be rejected by the server. These requests will be used as seeds for the specification requirement translator. To introduce potential processing differences between different HTTP servers, this invention also introduces common mutations into valid requests, such as header repetition, insertion of Unicode characters, header encoding, and case variations. Through these methods, this invention can also explore some subtle attack scenarios.

[0098] The specification requires the translator to translate the specification requirements previously extracted in the document analyzer module into test cases with assertions. If a protocol implementation violates an assertion during the testing phase, we consider the target implementation to have violated the specification. At this stage, the semantic definitions of the specification requirements need to be manually entered to help translate the requirements into test cases. This invention defines a series of message descriptions (e.g., valid, invalid, duplicate, empty, too long) and role actions (e.g., close connection, report error, respond with 200 status code, do not forward request), the former used to automatically generate test cases, and the latter used to determine subsequent difference tests.

[0099] Difference analysis tests multiple HTTP messages simultaneously within the same HTTP middleware, allowing for comparison of behaviors between tested programs. Examples include error messages, debug logs, rendered output, return values, and observed execution paths for each HTTP implementation. To facilitate analysis and evaluation of test results, an n-dimensional vector is first defined.

[0100]

[0101] The UUID is a unique number for each request, the status code is the HTTP middleware's response status code, the host represents the parsed result of the Host field in the request, and the data is the HTTP request body. In addition, users can define many other semantic information related to the HTTP protocol (such as HTTP version, HTTP method, etc.) to discover semantic gap vulnerabilities. This semantic information can be observed in various ways, including response data, error messages, and system logs. Under different detection models, users can define different detection rules based on HMetrics to discover semantic gap attacks. Furthermore, the Host value interpreted by the HTTP middleware differs from that of the backend server; in this respect, this invention outputs test cases as potential vulnerabilities, along with a description of the discovered vulnerabilities.

[0102] Based on the above embodiments, the present invention provides an example of using the above prediction method to complete a prediction. The present invention is implemented through the technical solutions described in steps S1 to S8 below:

[0103] S1: Retrieve multiple target HTTP middlewares to be processed;

[0104] S2: Extract specification requirements from RFC documents using a sentiment-based specification requirement finder;

[0105] S3: Use a text rule converter to extract and organize the natural language sentences in the RFC document through dependency analysis and textual implication, and transform the specification requirements into formal expressions;

[0106] S4: Use the ABNF rule extractor to extract valid ABNF rules from the RFC document;

[0107] S5: Use the ABNF rule adjuster to optimize and adjust the valid ABNF rules obtained by the ABNF rule adjuster to obtain a valid ABNF rule set;

[0108] S6: Use the ABNF rule generator to convert a valid ABNF rule set into an HTTP message;

[0109] S7: Use the specification-required translator to translate the HTTP message into a test case with assertions;

[0110] S8: Use test cases on the target HTTP middleware and analyze the differences in the results to summarize vulnerabilities.

[0111] In the above specific embodiments, the present invention provides a protocol inconsistency vulnerability mining method based on human-machine collaboration. This method involves acquiring multiple target middleware to be processed; testing each target middleware using pre-generated test samples to obtain test results; wherein the test samples are extracted and generated based on a pre-stored document analyzer; and analyzing each test result based on a pre-stored difference analysis algorithm to obtain analysis results. If the analysis results include preset differences, then a protocol inconsistency vulnerability is determined to exist among the target middleware. This invention, from a novel human-machine collaborative vulnerability mining perspective, targets HTTP middleware and utilizes a document analyzer to extract and generate test samples, along with differential testing methods, to semi-automatically mine protocol inconsistency vulnerabilities caused by semantic differences in various HTTP middleware. This achieves a systematic, scalable, more efficient, and more accurate method for mining protocol inconsistency vulnerabilities.

[0112] The protocol inconsistency vulnerability mining device based on human-machine collaboration provided by the present invention will be described below. The protocol inconsistency vulnerability mining device based on human-machine collaboration described below can be referred to in correspondence with the protocol inconsistency vulnerability mining method based on human-machine collaboration described above.

[0113] Figure 4 This is a schematic diagram of the protocol inconsistency vulnerability mining device based on human-machine collaboration provided by the present invention, as shown below. Figure 4 As shown, it includes a test file acquisition unit 410, an attack testing unit 420, a result analysis unit 430, and a result output unit 440, wherein,

[0114] The test file acquisition unit 410 is used to acquire multiple target middleware to be processed;

[0115] The attack testing unit 420 is used to test each of the target middlewares using pre-generated test samples to obtain test results; wherein, the test samples are extracted and generated based on a pre-stored document analyzer.

[0116] The result analysis unit 430 is used to analyze each of the test results based on a pre-stored difference analysis algorithm to obtain analysis results;

[0117] The result output unit 440 is used to determine, if the analysis results include preset differences, that there is a protocol inconsistency vulnerability among the target middleware.

[0118] Based on the above embodiments, in this device,

[0119] The pre-stored document analyzer includes:

[0120] A specification requirements finder, which is used to extract specification requirements from RFC documents;

[0121] A text rule converter, which is used to extract and organize natural language sentences in the RFC document through dependency analysis and textual entailment;

[0122] An ABNF rule extractor, which is used to extract valid ABNF rules from the RFC document;

[0123] The ABNF rule adjuster is used to optimize and adjust the valid ABNF rules obtained by the ABNF rule adjuster.

[0124] Based on the above embodiments, in this device,

[0125] Test samples are extracted and generated based on a pre-stored document analyzer, specifically including:

[0126] Extract grammatical rules, semantic information, valid ABNF rule sets, and sentiment analysis-based specification requirements from RFC documents;

[0127] The specification requirements are transformed into formal expressions through a text rule converter based on dependency analysis and textual implication.

[0128] The test case is generated based on the grammar rules, the semantic information, the valid ABNF rule set, and the formal expression.

[0129] Based on the above embodiments, in this device,

[0130] Extracting syntax rules and semantic information from RFC documents, specifically including:

[0131] Extract natural language sentences from RFC documents using a sentiment-driven, specification-requirement-based finder.

[0132] The natural language sentences are processed using a text rule converter to extract grammatical rules and semantic information from them.

[0133] Based on the above embodiments, in this device,

[0134] Extract the valid ABNF rule set from the RFC document, specifically including:

[0135] Extract valid ABNF rules from RFC documents;

[0136] The effective ABNF rules are optimized to obtain optimized ABNF rules;

[0137] The optimized ABNF rules are used as the effective ABNF rule set.

[0138] Based on the above embodiments, in this device,

[0139] Based on the grammar rules, the semantic information, the valid ABNF rule set, and the formal expression, the test cases are generated, specifically including:

[0140] Convert a valid ABNF rule set into an HTTP message;

[0141] Based on the specification requirements, the translator translates the HTTP message into a test case with assertions.

[0142] In the above specific embodiments, the present invention provides a protocol inconsistency vulnerability mining device based on human-machine collaboration. This device acquires multiple target middleware to be processed; tests each target middleware using pre-generated test samples to obtain test results; wherein the test samples are extracted and generated based on a pre-stored document analyzer; and analyzes each test result based on a pre-stored difference analysis algorithm to obtain analysis results. If the analysis results include preset differences, then a protocol inconsistency vulnerability is determined to exist among the target middleware. This invention, from a novel human-machine collaborative vulnerability mining perspective, targets HTTP middleware and utilizes a document analyzer to extract and generate test samples, along with differential testing methods, to semi-automatically mine protocol inconsistency vulnerabilities caused by semantic differences in various HTTP middleware. This achieves a systematic, scalable, more efficient, and more accurate method for mining protocol inconsistency vulnerabilities.

[0143] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a human-machine collaborative protocol inconsistency vulnerability mining method. This method includes: acquiring multiple target middleware to be processed; testing each target middleware using pre-generated test examples to obtain test results; wherein the test examples are extracted and generated based on a pre-stored document analyzer; analyzing each test result based on a pre-stored difference analysis algorithm to obtain analysis results; and determining that a protocol inconsistency vulnerability exists among the target middleware if the analysis results include preset differences.

[0144] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the protocol inconsistency vulnerability mining method based on human-computer collaboration provided by the above methods. The method includes: acquiring multiple target middleware to be processed; testing each of the target middleware using pre-generated test samples to obtain test results; wherein the test samples are extracted and generated based on a pre-stored document analyzer; analyzing each of the test results based on a pre-stored difference analysis algorithm to obtain analysis results; and determining that there is a protocol inconsistency vulnerability among the target middleware if the analysis results include a preset difference.

[0146] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the human-computer collaboration-based protocol inconsistency vulnerability mining method provided by the above methods. This method includes: acquiring multiple target middleware to be processed; testing each target middleware using pre-generated test samples to obtain test results; wherein the test samples are extracted and generated based on a pre-stored document analyzer; analyzing each test result based on a pre-stored difference analysis algorithm to obtain analysis results; and determining that a protocol inconsistency vulnerability exists among the target middleware if the analysis results include a preset difference.

[0147] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0148] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for discovering protocol inconsistency vulnerabilities based on human-machine collaboration, characterized in that, include: Acquire multiple target middleware to be processed; Each of the target middleware is tested using pre-generated test cases to obtain test results; wherein the test cases are extracted and generated based on a pre-stored document analyzer. The test results are analyzed based on the pre-stored difference analysis algorithm to obtain the analysis results; If the analysis results include preset differences, then it is determined that there is a protocol inconsistency vulnerability among the target middleware. Among them, multiple target middleware are programs with the same function but running in different HTTP middleware; The step of testing each target middleware using pre-generated test cases is as follows: each target middleware is tested independently using pre-generated test cases; The pre-stored document analyzer includes: A specification requirements finder, which is used to extract specification requirements from RFC documents; A text rule converter, which is used to extract and organize natural language sentences in the RFC document through dependency analysis and textual entailment; An ABNF rule extractor, which is used to extract valid ABNF rules from the RFC document; ABNF rule adjuster: The ABNF rule adjuster is used to optimize and adjust the valid ABNF rules obtained by the ABNF rule adjuster. Specifically, the extraction and generation of test samples based on a pre-stored document analyzer includes: Extract grammatical rules, semantic information, valid ABNF rule sets, and sentiment analysis-based specification requirements from RFC documents; The specification requirements are transformed into formal expressions through a text rule converter based on dependency analysis and textual implication. The test case is generated based on the grammar rules, the semantic information, the valid ABNF rule set, and the formal expression.

2. The method for discovering protocol inconsistency vulnerabilities based on human-machine collaboration according to claim 1, characterized in that, Extracting syntax rules and semantic information from RFC documents, specifically including: Extract natural language sentences from RFC documents using a sentiment-driven, specification-requirement-based finder. The natural language sentence is processed using a text rule converter to extract grammatical rules and semantic information from it.

3. The method for discovering protocol inconsistency vulnerabilities based on human-machine collaboration according to claim 1, characterized in that, Extract the valid ABNF rule set from the RFC document, specifically including: Extract valid ABNF rules from RFC documents; The effective ABNF rules are optimized to obtain optimized ABNF rules; The optimized ABNF rules are used as the effective ABNF rule set.

4. The method for discovering protocol inconsistency vulnerabilities based on human-machine collaboration according to claim 1, characterized in that, Based on the grammar rules, the semantic information, the valid ABNF rule set, and the formal expression, the test cases are generated, specifically including: Convert a valid ABNF rule set into an HTTP message; Based on the specification requirements, the translator translates the HTTP message into a test case with assertions.

5. A protocol inconsistency vulnerability discovery device based on human-machine collaboration, characterized in that, include: The test file acquisition unit is used to acquire multiple target middleware to be processed; An attack testing unit is used to test each of the target middlewares using pre-generated test samples to obtain test results; wherein the test samples are extracted and generated based on a pre-stored document analyzer. The result analysis unit is used to analyze each of the test results based on a pre-stored difference analysis algorithm to obtain the analysis results; The result output unit is used to determine, if the analysis results include preset differences, that there is a protocol inconsistency vulnerability among the target middleware; Among them, multiple target middleware are programs with the same function but running in different HTTP middleware; The step of testing each target middleware using pre-generated test cases is to test each target middleware independently using pre-generated test cases. The pre-stored document analyzer includes: A specification requirements finder, which is used to extract specification requirements from RFC documents; A text rule converter, which is used to extract and organize natural language sentences in the RFC document through dependency analysis and textual entailment; An ABNF rule extractor, which is used to extract valid ABNF rules from the RFC document; ABNF rule adjuster: The ABNF rule adjuster is used to optimize and adjust the valid ABNF rules obtained by the ABNF rule adjuster. Specifically, the extraction and generation of test samples based on a pre-stored document analyzer includes: Extract grammatical rules, semantic information, valid ABNF rule sets, and sentiment analysis-based specification requirements from RFC documents; The specification requirements are transformed into formal expressions through a text rule converter based on dependency analysis and textual implication. The test case is generated based on the grammar rules, the semantic information, the valid ABNF rule set, and the formal expression.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the protocol inconsistency vulnerability mining method based on human-machine collaboration as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the protocol inconsistency vulnerability mining method based on human-machine collaboration as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the protocol inconsistency vulnerability mining method based on human-machine collaboration as described in any one of claims 1 to 4.

Images