Industrial control protocol fuzz testing method and system based on large language model

Abstract

The invention belongs to the field of fuzz testing, and discloses an industrial control protocol fuzz testing method and system based on a large language model, and the method comprises the steps: obtaining a protocol document, a to-be-tested protocol code, and a protocol sample; analyzing the protocol document based on a large language model to obtain protocol analysis information; performing vulnerability extraction on the protocol sample with the vulnerability information based on a large language model to obtain a plurality of vulnerability documents; based on the retrieval enhancement generation framework, retrieving an external knowledge base by adopting the query statement and the protocol analysis information to generate an enriched initial seed set; performing code auditing on the protocol code to be tested based on the large language model to obtain a vulnerability code of the protocol code, and performing instrumentation on the vulnerability code of the protocol code to obtain an instrumented code; mutation is conducted on the input seeds based on a preset seed energy distribution rule, and fuzzy testing is conducted on the instrumentation codes based on the input seeds subjected to mutation. According to the invention, the probability and speed of generating abnormity in fuzzy testing are improved.

Description

Technical Field

[0001] The invention belongs to the field of fuzzy testing, and in particular relates to an industrial control protocol fuzzy testing method and system based on a large language model. Background Art

[0002] Industrial Control System (ICS), hereinafter referred to as ICS, is a general term for various control systems and related instruments used for process control. ICS protocols play an important role in the communication between system components. ICS protocols are used to communicate with Programmable Logic Controllers (PLC) or Remote Terminal Units (RTU) in industrial control systems. These ICS protocols face threats such as spoofing, replay, and denial of service (DoS) attacks. Fuzz testing, as an automated software testing technology, has become one of the most effective methods in the field of vulnerability discovery. It passes invalid, unexpected, or random data to the target program as test cases, and monitors the abnormal state of the target program during this period, trying to trigger the target vulnerability under test.

[0003] Currently, some researchers are studying how to improve the protocol fuzz testing method, as follows: In 2019, Luo et al. published "Polar: Function Code Aware Fuzz Testing of ICS Protocol", which improved AFL (American Fuzzy Lop, a coverage-guided fuzz testing tool developed by security researcher Michał Zalewski) for ICS protocol fuzz testing, called Polar-AFL. This method is based on static code analysis and dynamic taint analysis technology. First, the program code is compiled into an abstract syntax tree (AST), and the AST is traversed in depth first to find the vulnerabilities. Through dynamic taint analysis, the function code processing statements are automatically located, and the execution of these statements is tracked in the fuzz test.

[0004] However, this method has at least the following defects: 1. When locating vulnerabilities, this method can only find predefined vulnerabilities (such as writing memory, allocating memory, and other operations) by traversing AST to find predefined code statements, and it is very likely to miss other vulnerabilities in the code; 2. This method only considers the execution of vulnerable code, which may make it difficult to cover low-frequency paths and insufficient path exploration capabilities.

[0005] In 2020, Pham et al. published "Aflnet: A Greybox Fuzzer for Network Protocols", which extended AFL to perform fuzz testing on Internet protocols. This method captures the message sequence between the server and the client as the initial seed, identifies the server status by parsing the response code in the response message returned by the server, and retains those test cases that can effectively improve the code or state space coverage.

[0006] However, this method has the following limitations: 1. The captured initial seeds may not be diverse enough and cannot cover the server's state space well; 2. The message format of the protocol under test needs to be manually parsed to parse out the field where the response code is located; 3. It uses sockets to communicate with the server, which will greatly slow down the fuzz testing process; 4. It can only fuzz test protocols that have response codes in the response messages, such as HTTP (HyperText Transfer Protocol) and other Internet protocols, while many ICS protocols do not explicitly have response codes in the response messages.

[0007] In 2021, Lv et al. published "A Deep Convolution Generative Adversarial Networks Based Fuzzing Framework for Industry Control Protocols", which learned the ICS protocol message structure based on bidirectional long short-term memory (BILSTM) neural network and deep convolution generative adversarial network (DCGAN) and generated similar test cases without ICS protocol specifications.

[0008] This method has at least the following limitations: 1. This method has no feedback such as path coverage during fuzz testing, and cannot dynamically adjust strategies such as seed selection and mutation during fuzz testing, which makes the probability of fuzz testing discovering vulnerabilities low; 2. When monitoring program abnormalities, this method can only detect the abnormalities of the tested program through logs or manual observation, making it difficult to reproduce the discovered crashes.

[0009] Through the above analysis, the existing ICS protocol fuzz testing methods have at least the following problems in actual application: 1) Impact of invalid inputs: Due to the highly structured nature of the ICS protocol, test cases generated by random mutation usually contain a large number of invalid inputs; these invalid inputs seriously slow down the progress of fuzz testing and reduce the efficiency of testing; in addition, ICS protocols are usually not text-readable, so the structure-aware mutation method used on Internet protocols is almost infeasible on ICS protocols; 2) Complexity of generating test cases: The generation-based fuzz testing method requires parsing the protocol syntax structure and writing scripts to generate test cases in advance. This process is very time-consuming and labor-intensive, increasing the complexity and cost of testing. 3) Limitations of initial seeds: The effectiveness of fuzz testing methods and the speed of discovering vulnerabilities depend largely on the quality of the initial seeds. However, relying solely on pre-collected message sequences as initial seeds is difficult to cover the huge input structure diversity in the protocol specification, resulting in insufficient test coverage. 4) Limitations of coverage-guided fuzz testing: Vulnerabilities in programs are often caused by a very small amount of code, and existing methods usually only use coverage as an indicator to guide fuzz testing. This method may result in missing important code blocks that may cause vulnerabilities, affecting the comprehensiveness and depth of vulnerability detection. Summary of the invention

[0010] The purpose of the present invention is to provide an industrial control protocol fuzzy testing method and system based on a large language model, so as to solve the problems existing in the prior art, such as large amount of invalid input, high complexity of test cases, high limitation of initial seeds and high limitation of coverage orientation.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides an industrial control protocol fuzzy testing method based on a large language model, the method comprising: Obtain protocol documentation, protocol code to be tested, and protocol samples with vulnerability information; Parse the protocol document based on the large language model to obtain protocol parsing information; Based on the large language model, the vulnerability of the protocol samples with vulnerability information is extracted to obtain multiple vulnerability documents, and an external knowledge base is constructed with multiple vulnerability documents; Based on the retrieval enhancement generation framework, query statements and protocol parsing information are used to search the external knowledge base to generate an enriched initial seed set; Based on the large language model, the code of the protocol code to be tested is audited to obtain the vulnerable code of the protocol code, and the vulnerable code of the protocol code is plugged to obtain the plugged code; The initial seeds in the enriched initial seed set are used as the input seeds of the fuzz testing tool, and the input seeds are mutated based on the preset seed energy allocation rules, and the stub code is fuzz tested based on the mutated input seeds.

[0012] Preferably, the vulnerability document includes: a document number, a vulnerability description, vulnerability reproduction process information, and a vulnerability proof sample.

[0013] Preferably, the method further comprises: Segment each vulnerability document in the external knowledge base to obtain multiple paragraphs of each vulnerability document; Encode multiple paragraphs of each vulnerability document based on the pre-trained encoding model to obtain paragraph embedding representations corresponding to multiple paragraphs of each vulnerability document; Store all paragraph embedding representations into a pre-built vector database.

[0014] Preferably, the protocol parsing information at least includes: a protocol syntax format and a protocol function code; Based on the retrieval enhancement generation framework, the query statement and protocol parsing information are used to search the external knowledge base to generate an enriched initial seed set, including: Construct a Prompt template sequence based on query statements, protocol syntax format and protocol function code; Encode the query sentence based on the pre-trained encoding model to obtain the query embedding representation; Based on a preset search algorithm, searching in a vector database for k paragraph embedding representations whose similarity with the query embedding representation is greater than a preset similarity threshold, so as to take the k paragraph embedding representations whose similarity with the query embedding representation is greater than the preset similarity threshold as similar sequences, where k is a positive integer; The prompt template sequence is fused with similar sequences to obtain the input sequence; Input the input sequence into the retrieval-enhanced generation framework to obtain the initial seed; The query statement is updated to obtain an updated query statement, and the Prompt template sequence is reconstructed based on the updated query statement until a plurality of initial seeds are obtained, and the plurality of initial seeds constitute an enriched initial seed set.

[0015] Preferably, the protocol code to be tested is audited based on the large language model to obtain the vulnerable code of the protocol code, including: Determine whether the length of the protocol code to be tested exceeds a preset length, and if so, divide the protocol code to be tested into multiple code segments; Based on the thought chain prompt method, the large language model is guided to perform code audit on multiple code snippets to obtain audit results, which include: at least one vulnerable code in the multiple code snippets and a vulnerability score and a vulnerability cause corresponding to at least one vulnerable code, wherein the vulnerability score is used to characterize the possibility of a code snippet corresponding to the vulnerable code generating a vulnerability.

[0016] Preferably, the seed energy allocation rule is: Obtaining a vulnerability score corresponding to at least one vulnerable code in the multiple code fragments, and sorting the vulnerability score corresponding to the at least one vulnerable code to obtain a vulnerability score sequence; Based on a pre-built mapping function, mapping a vulnerability score corresponding to at least one vulnerable code to an executable body, wherein the executable body is built based on the vulnerability score sequence; Select any initial seed from the enriched initial seed set as the input seed, and determine the number of executions of the input seed on the execution body; Determine the original energy of the input seed based on a gray-box fuzz tester, wherein the original energy is used to determine the number of valid test cases generated after the input seed is mutated; Based on the original energy of the input seed and the number of executions of the input seed on the execution body, a piecewise increment function is constructed; Based on the piecewise incremental function, the vulnerability score value, the number of valid test cases generated after the input seed mutates, and the number of times the initial seed is selected, a seed energy allocation function is constructed, and the seed energy allocation function is used to allocate seed energy to each input seed; wherein the seed energy is used to determine the number of new test cases generated after the input seed mutates.

[0017] Preferably, the seed energy allocation function The expression is: ; In the formula, is the piecewise increment function, For the The initial seeds are selected as the number of executions of the input seeds on the execution body Y, For the The number of times the initial seeds are selected, For the The number of valid test cases generated after the initial seeds are selected as the input seeds are mutated. M is the preset energy upper limit value, To specify a constant, Select a function for the value.

[0018] Preferably, the method further comprises: In the fuzz testing of the instrumented code, the initial seeds in the enriched initial seed set are synchronously mutated based on the preset synchronous mutation rules to obtain the initial seeds after synchronous mutation, and the initial seeds after synchronous mutation are used as new input seeds of the fuzz testing tool.

[0019] Preferably, the synchronous mutation rule is: Select any initial seed from the enriched initial seed set to determine whether the initial seed has undergone deterministic mutation; If yes, obtain the field bit of the protocol function code corresponding to the initial seed; The field bits of the protocol function code corresponding to the initial seed are updated based on the legal candidate values ​​of the function code preset by the initial seed, so as to realize synchronous mutation of the initial seed.

[0020] In a second aspect, the present invention provides an industrial control protocol fuzzy testing system based on a large language model, the system is used to implement the above-mentioned industrial control protocol fuzzy testing method based on a large language model, the method comprising: The protocol acquisition module is used to obtain protocol documents, protocol codes to be tested, and protocol samples with vulnerability information; A protocol parsing module, used to parse the protocol document based on the large language model to obtain protocol parsing information, wherein the protocol parsing information at least includes: a protocol syntax format and a protocol function code; A vulnerability extraction module is used to extract vulnerabilities from protocol samples with vulnerability information based on a large language model, obtain multiple vulnerability documents, and construct an external knowledge base with multiple vulnerability documents; The seed enrichment module is used to search the external knowledge base based on the retrieval enhancement generation framework using query statements and protocol parsing information to generate an enriched initial seed set; The code audit module is used to perform code audit on the protocol code to be tested based on the large language model, obtain the vulnerable code of the protocol code, and insert the vulnerable code of the protocol code to obtain the inserted code; The fuzz testing module is used to use the initial seeds in the enriched initial seed set as the input seeds of the fuzz testing tool, mutate the input seeds based on the preset seed energy allocation rules, and perform fuzz testing on the stub code based on the mutated input seeds.

[0021] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned large language model-based industrial control protocol fuzzy testing method when executing the computer program.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned industrial control protocol fuzzy testing method based on a large language model.

[0023] Beneficial effects: 1. The present invention improves the effectiveness of fuzz testing by leveraging the powerful capabilities of large language models (LLMs) in natural language processing; 2. The present invention provides a protocol document, such as the ICS protocol, to a large language model (LLM) to automatically obtain information such as the protocol structure and protocol function code, without the need to manually analyze the protocol format. The acquired protocol knowledge can be used for initial seed enrichment and seed synchronization, greatly reducing the manual workload. 3. The present invention uses a large language model (LLM) to identify vulnerabilities in protocol samples (i.e., vulnerabilities exposed in the past by ICS protocol implementation software), and establishes an external knowledge base. The large language model is then used to enrich initial seeds to generate initial seeds that are similar to the characteristics of vulnerability proof samples in the vulnerability knowledge base. This method significantly improves the probability and speed of fuzz testing generating anomalies; 4. The present invention uses a large language model to locate the vulnerability information in the source code of the ICS protocol implementation software, and uses the seed energy allocation rule to allocate more energy to the seeds that execute these vulnerabilities during the fuzz testing process, so that they get more mutation opportunities, thereby exploring more paths. More path coverage means a higher probability of vulnerability discovery. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings: Figure 1 It is a flow chart of an industrial control protocol fuzzy testing method based on a large language model provided by an embodiment of the present invention; Figure 2 is an example diagram of a plugging method provided by an embodiment of the present invention; Figure 3 It is a block diagram of an industrial control protocol fuzzy testing system based on a large language model provided by an embodiment of the present invention; Figure 4 This is an experimental diagram of the path coverage of libmodbus fuzzy test for 5*24 hours provided by an embodiment of the present invention; Figure 5 This is an experimental diagram of the path coverage of libiec61850 fuzzy test for 5*24 hours provided by an embodiment of the present invention; Figure 6This is an experimental diagram of the path coverage number of 5*24 hours of lib60870 fuzz testing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0026] Terminology explanation: Fuzzing: Fuzzing is a dynamic testing process. The basic idea is to generate a large number of random inputs for testing the target program and observe the problems that arise during the testing process. The core of the whole process is how to effectively generate inputs to trigger problems.

[0027] AFL: AFL is an open source gray-box fuzz testing tool based on coverage guidance. It records the code coverage of samples and adjusts samples to improve coverage and increase the probability of discovering vulnerabilities. The AFL workflow is as follows: 1. Perform instrumentation when compiling the program from source code to record code coverage; 2. Select some input files and add them to the input queue as the initial test set; 3. Mutate the queue files according to a certain strategy; 4. If the coverage is updated after the mutated file, it is retained and added to the queue; 5. The above files will continue to loop, and the files that trigger the crash will be recorded.

[0028] Embodiment 1 Figure 1 FIG. 1 is a flow chart of an industrial control protocol fuzzy testing method based on a large language model provided by an embodiment of the present invention. Figure 1 As shown, this embodiment provides an industrial control protocol fuzzy testing method based on a large language model, the method comprising: Step S10: Acquire the protocol document, the protocol code to be tested, and the protocol sample with vulnerability information.

[0029] Step S20: Parse the protocol document based on the large language model to obtain protocol parsing information, which at least includes: a protocol syntax format and a protocol function code.

[0030] In this embodiment, the protocol document may be a protocol RFC document. RFC (Request For Comments) is a "Request for Comments" document that covers the standards and descriptions of various Internet technologies, protocols, and specifications; for example, Hypertext Transfer Protocol (HTTP) Protocol Version 1.1 (RFC 2616), which is as long as 176 pages. It is difficult to extract the ICS protocol structure and its function code from the RFC document using traditional manual parsing methods.

[0031] Therefore, this embodiment first collects public protocol RFC documents from the Internet, then designs prompts, and then drives LLM (Large Language Model) to parse these RFC documents, extracts the protocol syntax format and protocol function code from the RFC documents, and obtains protocol parsing information. Among them, prompt refers to: input text used to guide the large language model to generate specific outputs. Prompt can help set context, ask questions, or provide partial information so that the model can generate expected responses.

[0032] In this embodiment, in order to automatically parse the RFC document, it is necessary to let the LLM generate a response in a fixed format. The present invention provides the LLM with a template for generating a response, and allows the LLM to generate a response with a fixed template in JSON format. This embodiment provides a protocol document, such as the ICS protocol, to the large language model (LLM) to automatically obtain information such as the protocol structure and protocol function code, without the need to manually analyze the protocol format, and the obtained protocol knowledge can be used for initial seed enrichment and seed synchronization, greatly reducing the manual workload.

[0033] Step S30: extract vulnerabilities from protocol samples with vulnerability information based on the large language model to obtain multiple vulnerability documents, and construct an external knowledge base with the multiple vulnerability documents.

[0034] In this embodiment, ICS protocol implementation software vulnerability information is collected from public vulnerability libraries or open source communities as protocol samples with vulnerability information, and the protocol samples are integrated into a document. , then construct a prompt template and use LLM to analyze the document Distillation, which includes: Refining documents Text, and structured documents ; Specifically, the distilled document (That is, the vulnerability document) includes the following 4 parts: 1) Document number, each distilled document Each will be given a number; 2) Vulnerability description, documentation A brief description of the vulnerability; 3) Vulnerability reproduction process information: The specific reproduction process of the vulnerability; 4) Vulnerability proof example: can trigger the document The code or message of the vulnerability; Document Integrate into document library , Document Library As an external knowledge base, this embodiment establishes an external knowledge base, and then enriches the initial seeds through a large language model to generate initial seeds that are similar to the characteristics of vulnerability proof samples in the vulnerability knowledge base. This method significantly improves the probability and speed of fuzz testing generating anomalies.

[0035] As a further optimization of this embodiment, the method further includes: Step S301: segment each vulnerability document in the external knowledge base to obtain multiple paragraphs of each vulnerability document; the document segmentation steps for each vulnerability document are: segment the document base Each document in is segmented based on recursive character text, and the document Split into multiple paragraphs , so that later retrieval can be performed in a more fine-grained manner. Suppose the document Divided into paragraphs, where j=1,2,,…, J , which can be expressed as: .

[0036] Step S302: encoding multiple paragraphs of each vulnerable document based on the pre-trained encoding model to obtain paragraph embedding representations corresponding to the multiple paragraphs of each vulnerable document; That is, for each paragraph , using a pre-trained encoding model Generating paragraph embedding representations , which can be expressed as the following formula: .

[0037] In this embodiment, the coding model The BERT model (Bidirectional Encoder Representations from Transformers) is preferably used. The specific encoding process is: paragraph Segmented into word sequences ,in is the number of words. Then, each word Represented as a word embedding vector , where a=1,2,..,A. Then two special markers are used and to mark the beginning and end of the input sequence. The input sequence becomes: , in order to capture word order information, the position encoding Add to each word embedding vector: , To represent the word embedding vector after position encoding, it is then passed through the Transformer layer of the BERT model. The output of the layer is represented as: ; in, , It is a special token, usually placed at the beginning of the input sequence. Its main purpose is to provide a comprehensive representation of the entire input sequence after being processed by the BERT model. is another special token used to separate different sentences or fragments. In the input of BERT, a [SEP] token is usually added between sentences. It helps the model understand the structure and relationship of the sentences in the input; the output of the BERT model is the output of the last layer of Transformer encoder. In order to get the paragraph The overall vector representation of ,use The labeled vector representation is: ; in, is the total number of layers of the BERT model, It's the last layer Vector representation of the marker.

[0038] Step S303: store all paragraph embedding representations into a pre-built vector database.

[0039] In this embodiment, all paragraph embedding representations are stored in a vector database , used for subsequent retrieval, vector database It can be expressed as: ; In this example, Chroma is used as the vector database.

[0040] Step S40: Based on the search enhancement generation framework, the query statement and protocol parsing information are used to search the external knowledge base to generate an enriched initial seed set.

[0041] In this embodiment, the Retrieval-Augmented Generation (RAG) framework is an artificial intelligence technology that combines information retrieval technology with language generation models. Its main purpose is to enhance the processing capabilities of large language models (LLMs) by retrieving relevant information from external knowledge bases, especially when dealing with knowledge-intensive tasks such as question answering, text summarization, content generation, etc.

[0042] Specifically, based on the retrieval enhancement generation framework, the query statement and protocol parsing information are used to search the external knowledge base to generate an enriched initial seed set, including: Step S401: construct a Prompt template sequence based on the query statement, the protocol syntax format and the protocol function code; the Prompt template sequence T for: ; in, For the query statement, is the protocol syntax format, It is the protocol function code.

[0043] Step S402: Encode the query statement based on the pre-trained encoding model to obtain a query embedding representation, where the query embedding representation for: ; In this embodiment, the query statement is encoded using the encoding method of step S302 to obtain the query embedding representation .

[0044] Step S403: Based on a preset search algorithm, search in the vector database for k paragraph embedding representations whose similarity with the query embedding representation is greater than a preset similarity threshold, so as to take the k paragraph embedding representations whose similarity with the query embedding representation is greater than the preset similarity threshold as a similarity sequence, where k is a positive integer.

[0045] In this embodiment, the preset search algorithm adopts a vector similarity algorithm, that is, in the vector database Use a vector similarity algorithm to search for an embedding representation that matches the query Most similar The paragraph embedding representation , the corresponding paragraph is , which can be expressed as: ; In the formula, is the function of the vector similarity algorithm.

[0046] In this example, cosine similarity is used as the vector similarity algorithm, given the query embedding representation and vector database The paragraph embedding representation in , the cosine similarity calculation formula is as follows: ; in, is the dimension of the embedding vector, and They are query embedding representation and paragraph embedding representation in The components of a dimension.

[0047] Afterwards, all paragraphs are embedded according to the cosine similarity calculation results. Sort and select the one with the highest similarity Vectors: }; In the formula, }Will Sort the result set from largest to smallest, and then select the first k ones.

[0048] Step S404: Fusion the Prompt template sequence with the similar sequence to obtain an input sequence. for: .

[0049] Step S405: Input the input sequence into the search enhancement generation framework to obtain an initial seed. for: ; in, () Generative large language model in retrieval-enhanced generation framework , the generative large language model G can adopt the GPT-4o model.

[0050] Step S406: Update the query statement to obtain an updated query statement, and reconstruct the Prompt template sequence based on the updated query statement until multiple initial seeds are obtained, and the multiple initial seeds constitute an enriched initial seed set.

[0051] In this embodiment, each time the query statement is updated, the Prompt template sequence is reconstructed in step S401, and steps S402 to S404 are repeated again. This is repeated N times in total, and N initial seeds can be obtained, where N is a positive integer greater than 1. At this time, an enriched initial seed set W can be obtained: .

[0052] Step S50: Perform code audit on the protocol code to be tested based on the large language model to obtain the vulnerable code of the protocol code, and perform plugging on the vulnerable code of the protocol code to obtain the plugged code.

[0053] In this embodiment, the protocol code to be tested is the code of the ICS protocol implementation software. Specifically, based on the large language model, a code audit is performed on the protocol code to be tested to obtain the vulnerability code of the protocol code, including: Step S501: determine whether the length of the protocol code to be tested exceeds a preset length, and if so, divide the protocol code to be tested into multiple code segments; Set the code of ICS protocol implementation software DM The length is , the preset length is t, when >t, DM Split into B code segments with t as the maximum length : ; Where b=1,2,...,B, , For the rounding operation, each code snippet The length of does not exceed t.

[0054] Step S502: Based on the thought chain prompt method, guide the large language model to perform code audit on multiple code snippets to obtain audit results, wherein the audit results include: at least one vulnerable code in the multiple code snippets and a vulnerability score and a vulnerability cause corresponding to at least one vulnerable code, wherein the vulnerability score is used to characterize the possibility of a vulnerability in the code snippet corresponding to the vulnerable code. The large language model that performs code audit on multiple code snippets can adopt the GPT-4o model.

[0055] In this embodiment, a prompt template is constructed and a chain of thought prompting (COT) is used to gradually guide the large language model to think and reason. During code audit, the large language model will analyze multiple code snippets according to the prompt template, identify potential vulnerabilities in multiple code snippets, where vulnerabilities are more likely to occur, and output the corresponding audit results. The audit results are output in JSON format and include the following three parts: 1) Vulnerable code: Identified code snippets that may have vulnerabilities; 2) Vulnerability score: the evaluation score of the vulnerability by the large language model. The higher the vulnerability score, the more likely the large language model thinks that the vulnerability is likely to occur. 3) Causes of vulnerabilities: The large language model explains and analyzes the causes of vulnerabilities.

[0056] In this embodiment, the vulnerable code of the protocol code is inserted to obtain the inserted code, specifically: traversing the code of the ICS protocol implementation software DM , find the location of the vulnerable code in the code audit results, and insert a stub code after it , in the fuzz testing loop, each execution , The corresponding value will be increased by 1.

[0057] like Figure 2 As shown, Figure 2 In the code shown, The code is considered by LLM to be likely to cause a buffer overflow vulnerability because The function will The 4 bytes in the variable are copied to Unchecked when in variable and If the boundary or The actual size is smaller than , The function will access and write memory beyond the buffer.

[0058] Therefore, in The code is instrumented after it, and each time the code is executed during the fuzz testing process, the value of the corresponding pointer is increased by 1.

[0059] Step S60: using the initial seeds in the enriched initial seed set as input seeds of the fuzz testing tool, mutating the input seeds based on a preset seed energy allocation rule, and performing fuzz testing on the instrumented code based on the mutated input seeds.

[0060] In this embodiment, the seed energy allocation rule is: Step a10: obtaining a vulnerability point score corresponding to at least one vulnerable code in the multiple code fragments, and sorting the vulnerability point scores corresponding to the at least one vulnerable code to obtain a vulnerability score sequence; In this embodiment, the vulnerability scores can be sorted from large to small, and the vulnerability score sequence is: S={ };in, .

[0061] Step a20: Based on a pre-built mapping function, mapping the vulnerability score corresponding to at least one vulnerable code to an executable body, wherein the executable body is built based on the vulnerability score sequence; In this embodiment, three different buckets (i.e., execution body Y={ }) y1, y2 and y3, using the mapping function Will Mapped from high to low to three different buckets, y1 stores the first 10% of the fragile score sequence S, y2 stores the first 10% to 20% of the fragile score sequence S, and y3 stores the rest of the fragile score sequence S. When mapping, for each , first find its index position in S , according to the index position to determine the Which bucket is mapped to? ;in, v =1,2,…, V .

[0062] Step a30: Select any initial seed from the enriched initial seed set as an input seed, and determine the number of executions of the input seed on the execution body.

[0063] Step a40: Determine the original energy of the input seed based on the grey-box fuzz tester, where the original energy is used to determine the number of valid test cases generated after the input seed is mutated.

[0064] Step a50: constructing a piecewise increment function based on the original energy of the input seed and the number of executions of the input seed on the execution body; This embodiment constructs three piecewise increment functions based on three different buckets, which are: when If y1 is hit, the expression of the piecewise increment function is: ; In the formula, Represents a piecewise incremental function, which is used to calculate the x energy. The larger the value of x, The larger the x= , Indicates The initial seeds are selected as the number of executions of the input seeds on the execution body Y, where Y is y1, Indicates An initial seed is selected as the raw energy of the input seed, which is calculated by the traditional grey-box fuzz tester AFL's own algorithm.

[0065] when Hitting y2, the expression of the piecewise incremental function is: ; In the formula, x= , at this time, here Y is y2.

[0066] when Hitting y3, the expression of the piecewise increment function is: ; In the formula, x= , at this time, here Y is y3.

[0067] Step a60: Based on the piecewise incremental function, the vulnerability score value, the number of valid test cases generated after the input seed mutates, and the number of times the initial seed is selected, a seed energy allocation function is constructed, wherein the seed energy allocation function is used to allocate seed energy to each input seed; wherein the seed energy is used to determine the number of new test cases generated after the input seed mutates.

[0068] In this embodiment, the seed energy allocation function The expression is: ; In the formula, is the piecewise increment function, For the The initial seeds are selected as the number of executions of the input seeds on the execution body Y, For the The number of times the initial seeds are selected, For the The number of valid test cases generated after the initial seeds are selected as the input seeds are mutated. M is the preset energy upper limit value, To specify a constant, Select the function for the value, that is, from the second parameter M and the first parameter The smaller parameter.

[0069] In this embodiment, Is a predefined constant used to specify the number of The energy limit of , is more likely to be on the low-frequency path, as exponential growth, will execute the low frequency path Allocate more energy.

[0070] Seed energy refers to the amount of test resources allocated to each input sample (input seed) during fuzz testing. Specifically, seed energy determines how many new test cases a certain input sample will generate by mutation during the mutation phase. A high-energy seed will generate more mutations, giving it more chances to trigger new code paths and discover potential vulnerabilities.

[0071] Low-frequency paths refer to code execution paths that are less frequently triggered and explored during fuzz testing. The definition of low-frequency paths is as follows: ,in, is a specific path, Is the path The number of times it is triggered, is the total number of triggers of all paths. If If the value is small, then this path is considered a low-frequency path.

[0072] As a further optimization of this embodiment, during the fuzz testing process, seeds are selected from the seed queue (a queue composed of input seeds) for mutation. The present invention adds a new "synchronous" mutation strategy to the traditional gray box fuzz tester to synchronize useful mutation information between seeds with different function codes, and inputs with different function codes are used to synchronize useful mutation information between seeds with different function codes. This often leads to different execution paths, but there are many similar operations between some function codes. Synchronizing seed information can speed up the speed and probability of generating effective test cases.

[0073] Therefore, the method further comprises: In the fuzz testing of the instrumented code, the initial seeds in the enriched initial seed set are synchronously mutated based on the preset synchronous mutation rules to obtain the initial seeds after synchronous mutation, and the initial seeds after synchronous mutation are used as new input seeds of the fuzz testing tool.

[0074] Wherein, the synchronous mutation rule is: Step b10: Select any initial seed and determine whether the initial seed has undergone deterministic mutation. In AFL, when the initial seed is selected for the first time, it will first undergo deterministic mutation. Step b20: If yes, obtain the field bit of the protocol function code corresponding to the initial seed; Step b30: updating the field bits of the protocol function code corresponding to the initial seed based on the legal candidate value of the function code preset by the initial seed, so as to realize synchronous mutation of the initial seed.

[0075] In this embodiment, due to the similarity of some operations of different values ​​of the function code, the test cases generated at this stage meet many original constraints, and because the legitimate candidate values ​​of the function code are replaced, there is a high possibility of exploring new paths, which is more likely to trigger potential vulnerabilities.

[0076] The present invention has the following advantages: 1. Reduce manual workload: By using a large language model to automatically parse the protocol syntax format and obtain protocol function code knowledge, the need for manual analysis of protocol documents is greatly reduced, and work efficiency is improved; 2. Improve vulnerability mining efficiency: Use a large language model to enrich the initial seeds and generate initial seeds that are similar to the characteristics of vulnerability proof samples in the vulnerability knowledge base, which significantly improves the probability and speed of abnormal generation in fuzz testing.

[0077] 3. Enhanced path exploration capability: By allocating more energy to seeds that execute vulnerable code and low-frequency paths, the present invention significantly outperforms existing fuzz testing tools in path exploration, covers more paths, and discovers more potential vulnerabilities; 4. Ability to mine real-world vulnerabilities: Three CVE vulnerabilities have been discovered through the present invention, so the present invention has been proven to be effective in mining real-world ICS protocol implementation software vulnerabilities.

[0078] Embodiment 2 Figure 3 FIG. 1 is a block diagram of an industrial control protocol fuzzy testing system based on a large language model provided by an embodiment of the present invention. Figure 3 As shown, this embodiment provides an industrial control protocol fuzzy testing system based on a large language model, and the system is used to implement the industrial control protocol fuzzy testing method based on a large language model of embodiment 1, and the method includes: The protocol acquisition module is used to obtain protocol documents, protocol codes to be tested, and protocol samples with vulnerability information; A protocol parsing module, used to parse the protocol document based on the large language model to obtain protocol parsing information, wherein the protocol parsing information at least includes: a protocol syntax format and a protocol function code; A vulnerability extraction module is used to extract vulnerabilities from protocol samples with vulnerability information based on a large language model, obtain multiple vulnerability documents, and construct an external knowledge base with multiple vulnerability documents; The seed enrichment module is used to search the external knowledge base based on the retrieval enhancement generation framework using query statements and protocol parsing information to generate an enriched initial seed set; The code audit module is used to perform code audit on the protocol code to be tested based on the large language model, obtain the vulnerable code of the protocol code, and insert the vulnerable code of the protocol code to obtain the inserted code; The fuzz testing module is used to use the initial seeds in the enriched initial seed set as the input seeds of the fuzz testing tool, mutate the input seeds based on the preset seed energy allocation rules, and perform fuzz testing on the stub code based on the mutated input seeds.

[0079] This embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the industrial control protocol fuzzy testing method based on a large language model of the first embodiment is implemented.

[0080] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the industrial control protocol fuzzy testing method based on a large language model of the first embodiment is implemented.

[0081] The present invention has the following advantages: 1. Reduce manual workload: By using a large language model to automatically parse the protocol syntax format and obtain protocol function code knowledge, the need for manual analysis of protocol documents is greatly reduced, and work efficiency is improved; 2. Improve vulnerability mining efficiency: Use a large language model to enrich the initial seeds and generate initial seeds that are similar to the characteristics of vulnerability proof samples in the vulnerability knowledge base, which significantly improves the probability and speed of abnormal generation in fuzz testing.

[0082] 3. Enhanced path exploration capability: By allocating more energy to seeds that execute vulnerable points and low-frequency paths, the present invention significantly outperforms existing fuzz testing tools in path exploration, covers more paths, and discovers more potential vulnerabilities; 4. Ability to mine real-world vulnerabilities: Three CVE vulnerabilities have been discovered through the present invention, so the present invention has been proven to be effective in mining real-world ICS protocol implementation software vulnerabilities.

[0083] Embodiment 3 The industrial control protocol fuzz testing method based on a large language model in Example 1 is integrated into the fuzz testing tool AFL. In order to verify the effectiveness of Example 1, this example uses the real-world open source ICS protocol implementation software libmodbus, libiec61850, and lib60870 to evaluate the method of Example 1. The selected protocol implementation software information is shown in Table 1. The experiment also selected AFL and Polar-AFL as comparative experiments. This example runs a 24-hour fuzz test on each tested object, and repeats each experiment 5 times to establish statistical significance.

[0084] Table 1 Selected ICS protocol implementation software and their introduction

[0085] The experimental evaluation indicator is the fuzz test path coverage: comparing the changes in the path coverage of the three fuzz testing methods in the 5*24 hours experiment, the higher the path coverage means that the fuzz testing method can explore more paths.

[0086] The experimental results of fuzzy test path coverage are as follows Figure 4 , Figure 5 , Figure 6 As shown, Figure 4-Figure 6 In the figure, the horizontal axis Time is time, the vertical axis Total-paths is the total number of paths, experiment1 is the first repeated experiment, experiment2 is the second repeated experiment, experiment3 is the third repeated experiment, experiment4 is the fourth repeated experiment, and experiment5 is the fifth repeated experiment.

[0087] For the convenience of expression, MyFuzz is used to represent the present invention in the experiment. It can be seen from the figure that the present invention is better than the control groups AFL and Polar-AFL in the 24-hour fuzz test. On libmodbus, the present invention is about 3.57 times higher than AFL and about 3.34 times higher than Polar-AFL in 24-hour path coverage on average. On libiec61850, the present invention is about 2.72 times higher than AFL and about 2.19 times higher than Polar-AFL in 24-hour path coverage on average. On lib60870, the present invention is about 3.04 times higher than AFL and about 2.79 times higher than Polar-AFL on average.

[0088] At the same time, the present invention also discovered three unknown vulnerabilities on libmodbus, all of which have been reported to the developer and have obtained CVE numbers and have been confirmed as vulnerabilities. The vulnerability information discovered by the present invention is shown in Table 2.

[0089] Table 2 Vulnerabilities found by the present invention

[0090] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0091] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0092] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A fuzzy testing method for industrial control protocols based on a large language model, characterized in that: The method comprises: Obtain protocol documentation, protocol code to be tested, and protocol samples with vulnerability information; Parse the protocol document based on the large language model to obtain protocol parsing information; Based on the large language model, the vulnerability of the protocol samples with vulnerability information is extracted to obtain multiple vulnerability documents, and an external knowledge base is constructed with multiple vulnerability documents; Based on the retrieval enhancement generation framework, query statements and protocol parsing information are used to search the external knowledge base to generate an enriched initial seed set; Based on the large language model, the code of the protocol code to be tested is audited to obtain the vulnerable code of the protocol code, and the vulnerable code of the protocol code is plugged to obtain the plugged code; The initial seeds in the enriched initial seed set are used as the input seeds of the fuzz testing tool, and the input seeds are mutated based on the preset seed energy allocation rules, and the stub code is fuzz tested based on the mutated input seeds.

2. The industrial control protocol fuzzy testing method based on a large language model according to claim 1 is characterized in that: The vulnerability document includes: document number, vulnerability description, vulnerability reproduction process information and vulnerability proof sample.

3. The industrial control protocol fuzzy testing method based on a large language model according to claim 1 is characterized in that: The method further comprises: Segment each vulnerability document in the external knowledge base to obtain multiple paragraphs of each vulnerability document; Encode multiple paragraphs of each vulnerability document based on the pre-trained encoding model to obtain paragraph embedding representations corresponding to multiple paragraphs of each vulnerability document; Store all paragraph embedding representations into a pre-built vector database.

4. The industrial control protocol fuzzy testing method based on a large language model according to claim 3 is characterized in that: The protocol parsing information at least includes: protocol syntax format and protocol function code; Based on the retrieval enhancement generation framework, the query statement and protocol parsing information are used to search the external knowledge base to generate an enriched initial seed set, including: Construct a Prompt template sequence based on query statements, protocol syntax format and protocol function code; Encode the query sentence based on the pre-trained encoding model to obtain the query embedding representation; Based on a preset search algorithm, searching in a vector database for k paragraph embedding representations whose similarity with the query embedding representation is greater than a preset similarity threshold, so as to take the k paragraph embedding representations whose similarity with the query embedding representation is greater than the preset similarity threshold as similar sequences, where k is a positive integer; The prompt template sequence is fused with the similar sequence to obtain the input sequence; Input the input sequence into the retrieval-enhanced generation framework to obtain the initial seed; The query statement is updated to obtain an updated query statement, and the Prompt template sequence is reconstructed based on the updated query statement until a plurality of initial seeds are obtained, and the plurality of initial seeds constitute an enriched initial seed set.

5. The industrial control protocol fuzzy testing method based on a large language model according to claim 1 is characterized in that: Based on the large language model, the code of the protocol to be tested is audited to obtain the vulnerable code of the protocol code, including: Determine whether the length of the protocol code to be tested exceeds a preset length, and if so, divide the protocol code to be tested into multiple code segments; Based on the thought chain prompt method, the large language model is guided to perform code audit on multiple code snippets to obtain audit results, which include: at least one vulnerable code in the multiple code snippets and a vulnerability score and a vulnerability cause corresponding to at least one vulnerable code, wherein the vulnerability score is used to characterize the possibility of a code snippet corresponding to the vulnerable code generating a vulnerability.

6. The industrial control protocol fuzzy testing method based on a large language model according to claim 5 is characterized in that: The seed energy allocation rule is: Obtaining a vulnerability score corresponding to at least one vulnerable code in the multiple code fragments, and sorting the vulnerability score corresponding to the at least one vulnerable code to obtain a vulnerability score sequence; Based on a pre-built mapping function, mapping a vulnerability score corresponding to at least one vulnerable code to an executable body, wherein the executable body is built based on the vulnerability score sequence; Select any initial seed from the enriched initial seed set as the input seed, and determine the number of executions of the input seed on the execution body; Determine the original energy of the input seed based on a gray-box fuzz tester, wherein the original energy is used to determine the number of valid test cases generated after the input seed is mutated; Based on the original energy of the input seed and the number of executions of the input seed on the execution body, a piecewise increment function is constructed; Based on the piecewise incremental function, the vulnerability score value, the number of valid test cases generated after the input seed mutates, and the number of times the initial seed is selected, a seed energy allocation function is constructed, and the seed energy allocation function is used to allocate seed energy to each input seed; wherein the seed energy is used to determine the number of new test cases generated after the input seed mutates.

7. The industrial control protocol fuzzy testing method based on a large language model according to claim 6 is characterized in that: The seed energy allocation function The expression is: ； In the formula, is the piecewise increment function, For the The initial seeds are selected as the number of executions of the input seeds on the execution body Y, For the The number of times the initial seeds are selected, For the The number of valid test cases generated after the initial seeds are selected as the input seeds are mutated. M is the preset energy upper limit value, To specify a constant, Select a function for the value.

8. The industrial control protocol fuzzy testing method based on a large language model according to claim 4 is characterized in that: The method further comprises: In the fuzz testing of the instrumented code, the initial seeds in the enriched initial seed set are synchronously mutated based on the preset synchronous mutation rules to obtain the initial seeds after synchronous mutation, and the initial seeds after synchronous mutation are used as new input seeds of the fuzz testing tool.

9. The industrial control protocol fuzzy testing method based on a large language model according to claim 8 is characterized in that: The synchronous mutation rule is: Select any initial seed from the enriched initial seed set to determine whether the initial seed has undergone deterministic mutation; If yes, obtain the field bit of the protocol function code corresponding to the initial seed; The field bits of the protocol function code corresponding to the initial seed are updated based on the legal candidate values ​​of the function code preset by the initial seed, so as to realize synchronous mutation of the initial seed.

10. A large language model-based industrial control protocol fuzzy testing system, used to implement the large language model-based industrial control protocol fuzzy testing method described in any one of claims 1 to 9, characterized in that: The system comprises: The protocol acquisition module is used to obtain protocol documents, protocol codes to be tested, and protocol samples with vulnerability information; A protocol parsing module, used to parse the protocol document based on the large language model to obtain protocol parsing information, wherein the protocol parsing information at least includes: a protocol syntax format and a protocol function code; A vulnerability extraction module is used to extract vulnerabilities from protocol samples with vulnerability information based on a large language model, obtain multiple vulnerability documents, and construct an external knowledge base with multiple vulnerability documents; The seed enrichment module is used to search the external knowledge base based on the retrieval enhancement generation framework using query statements and protocol parsing information to generate an enriched initial seed set; The code audit module is used to perform code audit on the protocol code to be tested based on the large language model, obtain the vulnerable code of the protocol code, and insert the vulnerable code of the protocol code to obtain the inserted code; The fuzz testing module is used to use the initial seeds in the enriched initial seed set as the input seeds of the fuzz testing tool, mutate the input seeds based on the seed energy allocation rule, and perform fuzz testing on the stub code based on the mutated input seeds.

Images