Network protocol fuzz testing method based on large language model

Abstract

The invention discloses a network protocol fuzzy test method based on a large language model, which is characterized in that key information is automatically identified and extracted from a network protocol document by utilizing an advanced natural language processing technology, then a knowledge graph is constructed, and a fuzzy test driver is generated through cue word engineering on the basis of the graph. The method comprises the following steps: firstly, creating a network protocol knowledge graph, processing a related network protocol specification document by using a large model, creating references to all message type entities and protocol related states in the document, and then generating the network protocol knowledge graph by using the information through the large model; then, using the knowledge graph to create clusters from bottom to top, hierarchically organizing data into semantic clusters, and summarizing semantic concepts and topics in advance, so that comprehensive understanding of a network protocol is facilitated; the test driver code is then generated based on the prompt enhancement at the time of the query.

Description

Technical Field

[0001] The present invention relates to the field of network protocol automated testing, and in particular to a method for extracting key information from network protocol documents using a large language model and natural language processing technology, and generating a fuzzy test driver based on the information. Background Art

[0002] With the rapid development of network technology, network protocols are becoming increasingly complex. Traditional manual testing methods are no longer able to meet the needs of efficient and comprehensive testing. To improve the coverage and effectiveness of network protocol testing, researchers have been exploring automated testing technologies. However, existing automated testing methods often rely on predefined rules and patterns, lacking the ability to deeply understand and flexibly process protocol documentation.

[0003] It is against this backdrop that this paper proposes a network protocol fuzz testing method based on a large language model. This method automatically extracts key information from network protocol documents, constructs a knowledge graph, and generates fuzz testing drivers based on this graph. This improves the coverage and effectiveness of protocol testing, thereby providing stronger protection for network protocol security. Summary of the Invention

[0004] In order to solve the problems mentioned in the above background, the present invention combines natural language processing technology and network protocol fuzz testing method to provide a network protocol document knowledge graph extraction and fuzz test drive generation method based on a large language model.

[0005] This method first creates a network protocol knowledge graph. A large model is used to process relevant network protocol specification documents, creating references to all message type entities and protocol-related states in the documents. This information is then used to generate a network protocol knowledge graph from the large model. Semantic clustering is then performed, using this knowledge graph to create bottom-up clusters, hierarchically organizing the data into semantic clusters and pre-summarizing semantic concepts and topics to facilitate a comprehensive understanding of network protocols. These structures are then used to enhance query-time hints and provide material for protocol fuzz testing, assisting in test driver code generation. Finally, evaluation and verification are performed. The generated protocol test drivers are annotated with information indexes to prove that the answers are based on the network protocol specifications, enabling testers to quickly and accurately verify the driver code output against the original protocol documentation.

[0006] 1. A network protocol fuzz testing method based on a large language model, characterized by:

[0007] (1) Create references to the message type entity and the protocol-related state entity;

[0008] (2) hierarchically organized into semantic clusters about network protocols;

[0009] (3) Provide a preliminary summary of protocol semantic concepts and related topics;

[0010] (4) Generate fuzzy test code driver through prompt word engineering.

[0011] The network protocol fuzz testing method based on a large language model according to claim 1 is characterized in that the message type and protocol-related state entity extraction method in step (1) is as follows:

[0012] Leveraging natural language processing (NLP) techniques for named entity recognition (NER), we locate and categorize entities mentioned in network protocol documents. We apply pattern matching and regular expressions to identify message type definitions in specific formats, such as packet structures or command codes within protocols. By analyzing the grammatical structure and contextual information within the document, we extract the states and transition conditions of the protocol state machine. We build a prompt word project to describe and define protocol types and state machines, and use a large model to identify message type entities within network protocol documents.

[0013] 2. A network protocol fuzz testing method based on a large language model according to claim 1, characterized in that: the hierarchical organization into semantic clusters about the network protocol in step (2) is:

[0014] Semantic relationships are established between identified entities, including those related to data flow, function calls, and exception handling. Large language models are used to understand the contextual information in network protocol documents, accurately capturing the dependencies and interactions between different message entities. A hierarchical clustering algorithm is applied to organize entities with similar characteristics or relationships into groups. Clusters are automatically formed based on the semantic similarity of the entities to reveal the implicit structure and patterns in the protocol documents. Topic modeling techniques are then used to further refine and identify the main themes within the clusters. Topics are used to represent the core concepts and functional modules of the network protocol, and a hierarchical structure is constructed to reflect protocol elements at different levels of abstraction.

[0015] 3. A network protocol fuzz testing method based on a large language model according to claim 1, characterized in that: the method for generating fuzz testing code by prompt word engineering in step (3) is as follows:

[0016] Using natural language processing technology, we construct a prompt word project to identify key topics within the document. These topics typically reflect the protocol's primary functions and application scenarios. We then associate the identified key concepts with related topics to construct a concept-topic network. This network helps us understand how different concepts organize and interact around a specific topic. Based on this constructed graph, we automatically generate a summary of the protocol's semantic concepts and related topics. This summary, presented in natural language, summarizes the protocol's key features, functions, and application scenarios.

[0017] 4. A network protocol fuzz testing method based on a large language model according to claim 1, characterized in that: the process of generating fuzz test code by prompt word engineering in step (4) is as follows:

[0018] First, we developed fuzz testing drivers for stateful protocols such as FTP, RTSP, and SMTP. Using the Boofuzz ​​network protocol fuzz testing framework, we manually read protocol documents and extracted message types and protocol state transition relationships. We also extracted key global variables maintained in the network protocol implementation and the processing of these global variables during message transfer. Based on this information, we constructed prompt words, thereby building a prompt word database for stateful protocol fuzz testing drivers.

[0019] Next, traffic for the target test protocol is collected and extracted from the traffic file in JSON format. Traffic information at the transport layer and below is removed, while application-layer traffic is specifically preserved. Subsequently, using the network protocol knowledge graph generated in the previous steps, a prompt word query is constructed. This allows for semantic clustering of the original corpus related to protocol state transitions, and the contextual corpus information required for fuzz testing is used to construct a prompt word template.

[0020] Finally, a workflow is constructed based on the network protocol fuzz test driver generation task. The prompt word template is input multiple times by querying the large language model. The strategy of first generating the driver code framework and then completing it is adopted to generate message type test code, protocol syntax tree state transfer code, and protocol state transfer reception code in sequence. The generated driver is scored for a single message type to evaluate the driver generation effect. Multiple iterations are performed and the best one is selected as the fuzz test driver for the target protocol.

[0021] The beneficial effects of the above technical solution of the present invention are as follows:

[0022] By utilizing large language models and natural language processing technology, the present invention realizes the automated knowledge graph extraction of network protocol documents, reduces manual intervention, improves efficiency, and increases the degree of test automation. At the same time, based on knowledge graphs and semantic clustering, the present invention can generate fuzzy test cases covering various states and transitions of network protocols, thereby enhancing the comprehensiveness of the test and enhancing the test coverage. Moreover, by pre-summarizing the semantic concepts and related topics of the protocol, the present invention can generate more accurate and targeted test cases, thereby improving the effectiveness of the test and enhancing the quality of the test cases. The test-driven code generated in the present invention will be annotated with information indexes to ensure the consistency of the test cases with the original protocol documents, making it easier for testers to verify and review, thereby effectively ensuring the accuracy of the test. The constructed network protocol knowledge graph can be used as a centralized storage of network protocol knowledge, facilitating the retrieval, updating, and reuse of knowledge, and providing support for continuous testing activities. Through summaries and descriptions generated in natural language, the present invention helps testers better understand complex network protocol documents, lowers the threshold for understanding, and improves the comprehensibility of the documents.

[0023] The method of the present invention can be easily integrated into the continuous integration / continuous testing (CI / CT) process to achieve continuous testing and evaluation of network protocols, while supporting continuous integration and continuous testing. The method of the present invention is not limited to a specific network protocol, has good adaptability, can be extended to a variety of different network protocols and application scenarios, and has good adaptability. Through automated and intelligent test-driven generation, the present invention reduces the workload of manually writing test cases and reduces testing costs. Through comprehensive and in-depth testing, the present invention helps to discover potential security vulnerabilities in network protocols, thereby improving the security of network communications.

[0024] In summary, the present invention provides an innovative network protocol testing method by combining a large language model with fuzz testing technology, which not only improves the efficiency and quality of the test, but also provides a strong guarantee for the security and reliability of the network protocol. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a network protocol fuzz testing method based on a large language model of the present invention;

[0026] Figure 2 Constructing a protocol knowledge graph using network protocol documents through community detection algorithms

[0027] Figure 3 Hierarchically organize the network protocol semantic clustering flow chart

[0028] Figure 4 Generate fuzz test code driving flowchart through prompt word engineering DETAILED DESCRIPTION

[0029] 1. Create references to the message type entity and the protocol-related state entity:

[0030] (1) Use pre-trained NLP models or custom trained models to perform NER tasks, identifying and classifying key entities in network protocol documents, such as protocol names, packet types, and state names. Define pattern matching rules, use regular expressions to match text in specific formats, and apply these rules to extract information from documents that conform to specific patterns. Identify message type definitions in specific formats within documents, such as packet structures or command codes. Analyze the grammatical structure of the document, such as conditional statements and flow charts, and extract the various states and transition conditions of the protocol state machine. By leveraging NLP technology to understand contextual information, accurately extract states and transition conditions, and thus implement protocol grammatical structure and context analysis.

[0031] (2) Create a set of prompt words or descriptors that accurately describe the key characteristics of the protocol. These prompt words will be used to guide the large model to more accurately identify and understand the content of network protocol documents. Based on this prompt word engineering, it provides descriptions and definitions for protocol types and state machines. Subsequently, the document content processed by NER, pattern matching, and prompt word engineering is input into the large language model, which processes and analyzes the document content to identify message type entities.

[0032] 2. Hierarchically organized into semantic clusters about network protocols:

[0033] (1) Use NLP technology to identify dependencies and interactions between entities, and identify semantic relationships between entities, such as data flow, function calls, exception handling, etc., to establish semantic relationships between entities. Use large language models to understand contextual information, analyze model outputs, determine semantic connections between entities, capture complex relationships between entities, complete the understanding of protocol context semantics, and accurately capture the dependencies and interactions between different types of message entities.

[0034] (2) Define similarity metrics for clustering algorithms, such as semantic similarity and structural similarity, apply a hierarchical clustering algorithm, and automatically form clusters based on the defined similarity metrics, organizing message type entities with similar characteristics or relationships into groups. Apply the topic modeling technique Latent Dirichlet Allocation (LDA) to identify the main topics in each cluster, which represent the core concepts and functional modules of the network protocol.

[0035] (3) Based on the results of topic modeling, construct a hierarchical knowledge structure and ensure that the structure can reflect the hierarchical relationship from high-level concepts to specific implementations. Use topic modeling to construct a hierarchy that reflects protocol elements at different levels of abstraction. Verify whether the clustering results are consistent with the actual structure and function of the network protocol. Adjust the clustering algorithm and topic modeling parameters as needed to ensure the accuracy and effectiveness of the clustering and hierarchical structure, ensure high-quality extraction of the network protocol knowledge graph, and document the clustering and hierarchical process so that other developers and researchers can understand and use it.

[0036] 3. Generate fuzz test code driver through prompt word engineering:

[0037] (1) First, professional protocol testers manually read the protocol documents, extract the message types and protocol state transition relationships, extract the global key variables maintained in the network protocol implementation and their processing processes, and then use the Boofuzz ​​network protocol fuzz testing framework to write fuzz test drivers for stateful protocols such as FTP, RTSP, and SMTP. Based on the extracted protocol information, prompt words are constructed to build a prompt word database for stateful protocol fuzz test drivers, which is used for subsequent fuzz test driver generation.

[0038] (2) Collect traffic data of the target protocol, convert the traffic file into JSON format, and then remove the information at the transport layer and below, retaining the application layer traffic information. Use the network protocol knowledge graph to build a prompt word query, obtain the original corpus related to the protocol state transition from the semantic clustering, complete the construction of the prompt word query and the establishment of the test driver template, and prepare the context corpus information for generating the fuzz test driver.

[0039] (3) Define the network protocol fuzz test driver generation workflow, including generating the driver code framework and completion strategy, using multiple iterations to gradually generate and iteratively improve the test code, and identify the traffic information generated by the test driver based on the existing traffic analysis tool, while also scoring the test level, and further optimizing the driver iteration through expert prompts from the traffic analysis tool. The specific generation process includes: generating message type test code, protocol syntax tree state transition code, and protocol state transition reception code. Based on the initially generated driver code framework, generative completion is gradually performed, and the generated driver is scored for a single message type to evaluate the driver generation effect. Perform multiple iterations and select the optimal driver as the fuzz test driver for the target protocol.

[0040] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A network protocol fuzz testing method based on a large language model, characterized by: (1) Create references to the message type entity and the protocol-related state entity; (2) hierarchically organized into semantic clusters about network protocols; (3) Provide a preliminary summary of protocol semantic concepts and related topics; (4) Generate fuzzy test code driver through prompt word engineering.

2. A network protocol fuzz testing method based on a large language model according to claim 1, characterized in that: The message type and protocol-related state entity extraction method in step (1) is as follows: Leveraging natural language processing (NLP) techniques for named entity recognition (NER), we locate and categorize entities mentioned in network protocol documents. We apply pattern matching and regular expressions to identify message type definitions in specific formats, such as packet structures or command codes within protocols. By analyzing the grammatical structure and contextual information within the document, we extract the states and transition conditions of the protocol state machine. We build a prompt word project to describe and define protocol types and state machines, and use a large model to identify message type entities within network protocol documents.

3. The network protocol fuzz testing method based on a large language model according to claim 1, characterized in that: The hierarchical organization in step (2) into semantic clusters about network protocols: Establish semantic relationships between identified entities, including data flow, function calls, exception handling, etc. Leveraging large language models to understand the contextual information within network protocol documents, the team accurately captures the dependencies and interactions between entities of different message types. A hierarchical clustering algorithm is applied to group entities with similar characteristics or relationships. Clusters are automatically formed based on semantic similarity to reveal the underlying structure and patterns within protocol documents. Topic modeling techniques are then employed to further refine and identify key themes within the clusters. Topics represent the core concepts and functional modules of network protocols, and a hierarchical structure is constructed to reflect protocol elements at different levels of abstraction.

4. The network protocol fuzz testing method based on a large language model according to claim 1, characterized in that: The method for generating fuzzy test code driving by prompt word engineering in step (3) is as follows: Using natural language processing technology, we construct a prompt word project to identify key topics within the document. These topics typically reflect the protocol's primary functions and application scenarios. We then associate the identified key concepts with related topics to construct a concept-topic network. This network helps us understand how different concepts organize and interact around a specific topic. Based on this constructed graph, we automatically generate a summary of the protocol's semantic concepts and related topics. This summary, presented in natural language, summarizes the protocol's key features, functions, and application scenarios.

5. The network protocol fuzz testing method based on a large language model according to claim 1, characterized in that: The process of generating fuzzy test code by prompt word engineering in step (4) is as follows: First, we developed fuzz testing drivers for stateful protocols such as FTP, RTSP, and SMTP. Using the Boofuzz ​​network protocol fuzz testing framework, we manually read protocol documents and extracted message types and protocol state transition relationships. We also extracted key global variables maintained in the network protocol implementation and the processing of these global variables during message transfer. Based on this information, we constructed prompt words, thereby building a prompt word database for stateful protocol fuzz testing drivers. Next, traffic for the target test protocol is collected and extracted from the traffic file in JSON format. Traffic information at the transport layer and below is removed, while application-layer traffic is specifically preserved. Subsequently, using the network protocol knowledge graph generated in the previous steps, a prompt word query is constructed. This allows for semantic clustering of the original corpus related to protocol state transitions, and the contextual corpus information required for fuzz testing is used to construct a prompt word template. Finally, a workflow is constructed based on the network protocol fuzz test driver generation task. The prompt word template is input multiple times by querying the large language model. The strategy of first generating the driver code framework and then completing it is adopted to generate message type test code, protocol syntax tree state transfer code, and protocol state transfer reception code in sequence. The generated driver is scored for a single message type to evaluate the driver generation effect. Multiple iterations are performed and the best one is selected as the fuzz test driver for the target protocol.

Images