Large language model-based stateful Internet of Things equipment fuzz testing method and apparatus

Abstract

The invention discloses a stateful Internet of Things equipment fuzz testing method and device based on a large language model. The method comprises the following steps: receiving protocol historical data; analyzing the protocol historical data by using a protocol analysis engine driven by a large oracle model to obtain a protocol field capable of representing a system state; the protocol fields are expressed as state nodes in a standardized mode; and combining the state nodes into a state diagram, judging a system state triggered by the fuzzy test according to the state diagram, and guiding the fuzzy test to be carried out in an untriggered state.

Description

Technical Field

[0001] This application relates to the technical field of protocol security testing, and particularly to a stateful Internet of Things device fuzz testing method and apparatus based on a large language model. Background Art

[0002] Internet of Things (IoT) devices refer to intelligent devices that can collect, send, and receive data through the Internet connection. They can include household appliances, industrial equipment, wearable devices, sensors, etc. The core purpose of IoT devices is to achieve automation, improve efficiency and convenience through the interconnection and interoperability of devices. However, the diversity and complexity of IoT devices pose many technical challenges.

[0003] The internal state of IoT devices can be divided into multiple aspects, including hardware state, software state, network state, and application state, etc. The hardware state involves the working state of sensors, the signal strength of communication modules, power supply, and storage state; the software state includes firmware version, error logs, and configuration parameters; the network state includes the connection status of the device, IP address, data transmission rate, etc.; the application state reflects the current task execution status of the device and whether it responds to user requests. A comprehensive understanding of these internal states is crucial for ensuring the stability and security of IoT devices.

[0004] In current technologies, IoT devices are often exposed to untrusted network environments, facing potential security risks and stability problems. To ensure the security and reliability of devices, fuzz testing, as an effective vulnerability discovery technology, is widely used. By inputting random or abnormal data into the device, fuzz testing can help discover problems such as crashes, data leaks, and security vulnerabilities that may occur when the device processes invalid or abnormal inputs. Fuzz testing can not only discover the robustness problems of the device but also reveal firmware vulnerabilities and security problems related to network interfaces.

[0005] However, existing IoT device fuzz testing methods still have some deficiencies. First, due to the wide variety of IoT devices, traditional fuzz testing methods often struggle to adapt to the complex protocols and state changes of different devices, resulting in incomplete test coverage. Second, traditional fuzz testing methods have insufficient ability to represent device states, often ignoring the complex behaviors of devices in multiple states and making it difficult to comprehensively evaluate the security and stability of devices. These problems limit the effectiveness and efficiency of fuzz testing. Therefore, how to improve the test efficiency, accurately represent the internal state of the device, and ensure coverage of all potential device states has become an urgent problem in current technologies. Summary of the Invention

[0006] The objective of the embodiments of this application is to provide a method and device for fuzz testing stateful Internet of Things devices based on large language models, so as to solve the problems of low testing efficiency and insufficient device state characterization in the prior art, thereby improving the fuzz testing efficiency and more accurately reflecting the internal system state of the device.

[0007] According to the first aspect of the embodiments of this application, there is provided a method for fuzz testing stateful Internet of Things devices based on large language models, including:

[0008] Receiving protocol historical data;

[0009] Using a protocol analysis engine driven by a large prediction model to parse the protocol historical data to obtain protocol fields that can represent the system state;

[0010] Normalizing the protocol fields into state nodes;

[0011] Combining the state nodes into a state graph, and based on this, judging the system states triggered by fuzz testing, and guiding the fuzz testing with the untriggered states.

[0012] According to the second aspect of the embodiments of this application, there is provided a device for fuzz testing stateful Internet of Things devices based on large language models, including:

[0013] A receiving module, configured to receive protocol historical data;

[0014] A parsing module, configured to use a protocol analysis engine driven by a large prediction model to parse the protocol historical data to obtain protocol fields that can represent the system state;

[0015] A normalization module, configured to normalize the protocol fields into state nodes;

[0016] A fuzz testing module, configured to combine the state nodes into a state graph, and based on this, judge the system states triggered by fuzz testing, and guide the fuzz testing with the untriggered states.

[0017] According to the third aspect of the embodiments of this application, there is provided an electronic device, including:

[0018] One or more processors;

[0019] A memory, configured to store one or more programs;

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect.

[0021] According to a fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium having computer instructions stored thereon, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0022] The technical solutions provided by the embodiments of the present application may include the following beneficial effects:

[0023] As can be seen from the above embodiments, the present application uses a protocol analysis engine based on a large language model to parse the protocol historical data of Internet of Things devices, and normalizes the extracted protocol fields into state nodes, thereby constructing a state diagram, solving the problems of low efficiency of fuzz testing and insufficient representation of device states in the prior art. Therefore, it can intelligently generate targeted test inputs, improve the coverage and accuracy of fuzz testing, and ensure a comprehensive representation of the internal system state of the device. Furthermore, it significantly improves the security and robustness of Internet of Things devices in complex interaction environments, optimizes the testing process, and reduces testing time and resource consumption.

[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings here are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0026] Figure 1 is a flowchart of a method for fuzz testing stateful Internet of Things devices based on a large language model shown according to an exemplary embodiment.

[0027] Figure 2 is a block diagram of a device for fuzz testing stateful Internet of Things devices based on a large language model shown according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0029] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0031] Figure 1 is a flowchart of a stateful Internet of Things device fuzz testing method based on a large language model shown according to an exemplary embodiment, as Figure 1 shown, this method is applied to a terminal and may include the following steps:

[0032] S1: Receive protocol historical data;

[0033] Specifically, first collect protocol historical data from the interactions between the Internet of Things device and external systems, users, or other devices. These data usually include the request messages and response messages of the device. Connect through the communication interface of the device (such as Wi-Fi, Bluetooth, etc.), and use the supporting interaction software or packet capture tool to intercept the protocol data during the device communication process. The collected data format can be common formats such as JSON, XML, etc. to ensure the integrity and accuracy of the data. This design ensures that the data used in subsequent analysis is real, comprehensive, and can accurately reflect the behavior and state of the device during actual operation, thus providing a reliable basis for subsequent state analysis and fuzz testing, and improving the test coverage and accuracy.

[0034] S2: Use a protocol analysis engine driven by a large prediction model to parse the protocol historical data to obtain protocol fields that can represent the system state; this step includes the following sub-steps:

[0035] S21: According to the Internet of Things device to be tested, install and use the supporting interaction software, interact with the device function by function and intercept the messages;

[0036] Specifically, first select and install the supporting interactive software according to the Internet of Things device to be tested. The software can simulate the operations of the device and communicate with the device. By interacting with each function of the device one by one, the software intercepts the corresponding request messages and response messages, which contain the key fields of the device status. In this way, the protocol historical data of the device in actual use can be obtained, ensuring the comprehensiveness, authenticity, and reliability of the data. This design ensures that the data used in subsequent analysis reflects the actual operating conditions of the device and can provide accurate input for the large language model-driven protocol analysis, thereby improving the coverage and accuracy of fuzz testing.

[0037] S22: Invoke the open API of the large language model, send the predefined prompt template to the large language model, and obtain a response;

[0038] Specifically, invoke the open API of the large language model, and send the predefined prompt template to the model. The template is designed according to the device protocol structure to help the model identify and extract the key fields in the protocol message, such as device commands and response results. In this way, the large language model can parse the device protocol data and return the protocol fields containing the device status information, providing a basis for the subsequent construction of the state diagram. Such a design can improve the efficiency and accuracy of protocol parsing. The automated parsing process reduces manual intervention, and at the same time improves the flexibility and adaptability of the method, ensuring that it can handle multiple device protocols.

[0039] The prompt template consists of two parts. First, input the protocol message without any prompts, allowing the large language model to identify the protocol type. Next, the prompt will require the large language model to extract the fields in the message that can represent the device system status.

[0040] S23: Extract the status fields from the large language model response;

[0041] Specifically, after the large language model returns a response, extract the key information containing the current status of the device from the response, such as the execution result of the device command and the status of the device function. By parsing the data structure returned by the large language model, identify the fields that identify the device status, such as the "method" field (indicating a device function request) and the "result" or "params" field (indicating the response result or device parameters). This design can ensure that the status changes that occur during the interaction of the device are accurately captured, thereby providing reliable status nodes for subsequent fuzz testing. In this way, the device status can be automatically extracted, improving the analysis efficiency, reducing manual intervention, and at the same time ensuring the accuracy and comprehensiveness of the extracted information.

[0042] The status fields in the large language model response include two parts, the fields of the request message and the fields of the response message.

[0043] When the IoT devices interact, they use protocol messages in JSON format. The fields representing the system state extracted by the large language model are usually the "method" field in the request and the "result" or "params" field in the response. The "method" field represents the function requested by the device. Combining the "method" field with the "result" or "params" field in the response forms a state node, which is used to represent the current internal state of the device. The "result" or "params" field represents whether the device has successfully executed this function and the related parameters returned.

[0044] S3: Normalize the protocol fields into state nodes;

[0045] Specifically, in this step, the protocol fields extracted from the large language model are normalized and uniformly formatted into state nodes. A state node is an abstract representation of the system state of a device at a certain moment, including key information such as the device's request command and response result. For example, combining the "method" field (representing the function requested by the device) with the "result" or "params" field (representing the execution result or related parameters of the response) into a state node to describe the state of the device after a specific interaction. The purpose of this design is to unify the fields in different devices or protocol formats through normalization processing, simplify the subsequent analysis process, and ensure the accurate identification and representation of the device state. By this method, the recognition accuracy and efficiency of the device state during fuzz testing can be improved, while ensuring compatibility between different devices and reducing possible errors during the processing.

[0046] S4: Combine the state nodes into a state diagram and use it to judge the system states triggered by fuzz testing, and use the untriggered states to guide the fuzz testing; this step includes the following sub-steps:

[0047] S41: Collect the normalized state nodes to form a state diagram;

[0048] Specifically, in this step, the normalized state nodes obtained from the previous steps are combined into a state diagram according to the logic and process of device interaction. The state diagram shows the conversion relationship between different states of the device by connecting the state nodes to each other. For example, the conversion of the device from a function request state to a response result state can be represented by an edge connecting two state nodes, thus forming the state diagram of the device. The purpose of this design is to clearly display the various states of the device and their mutual relationships in a graphical way, facilitating the tracking and identification of changes in different states during the subsequent fuzz testing process. By constructing the state diagram, the state changes of the device in different interaction scenarios can be intuitively understood, improving the coverage rate and effectiveness of fuzz testing.

[0049] S42: Start fuzz testing, mutate the parameters of the message, and collect the responses of the IoT device to the mutated message;

[0050] Specifically, mutating the parameters of the message includes: Mutating the parameters of the message is divided into two categories. One is for a single message, including adding, deleting strings, and replacing boundary values and special values. The other is for a message sequence, which involves shuffling the order of the message sequence or copying a certain message. In this step, fuzz testing generates new test cases by mutating the parameters of the previously collected protocol messages. The mutation methods can include randomly modifying the data in the message, replacing boundary values, adding or deleting fields, etc. Then, the mutated message is sent to the IoT device, and the responses of the device to these mutated messages are recorded. For example, it may be observed whether the device crashes, returns abnormal data, or exhibits other unexpected behaviors. The purpose of this design is to test the robustness and stability of the device when facing unforeseen inputs by simulating various abnormal inputs. In this way, potential security vulnerabilities and stability issues of the device can be effectively revealed. The advantage of this design is that it can cover a variety of potential attack scenarios, ensure that the device can still operate efficiently and securely when facing abnormal inputs, and thus improve the security and robustness of the device.

[0051] S43: The appearance of a new response means the emergence of a new state node, and the new state node is merged into the state diagram;

[0052] Specifically, in this step, after the mutated message generated by fuzz testing is sent to the IoT device, the device will generate responses according to the different messages. If the response of the device does not match the state nodes in the existing state diagram, it means that the device has entered a new state. At this time, the state fields corresponding to the response are extracted to form a new state node, which is then added to the state diagram. This process can continue until no new responses appear. The purpose of designing this step is to ensure that the test covers all possible device states by dynamically updating the state diagram, avoiding missing any potential abnormal states or vulnerabilities. The advantage of this design is that by continuously expanding the state diagram, it can accurately reflect the actual behavior of the device when facing various inputs, improve the comprehensiveness and effectiveness of the test, and at the same time help developers better understand the state transition process of the device, so as to optimize the security and stability of the device.

[0053] S44: Record the mutation strategy corresponding to the triggered new state node, and mutate the seed using the mutation strategy;

[0054] Specifically, in this step, whenever a new state node is added to the state diagram, the system records the mutation strategies used to trigger this new state node. These mutation strategies include specific mutation methods for message parameters, such as modifying the values of certain fields, adding or deleting fields, changing the order of fields, etc. Next, based on the recorded mutation strategies, the system further mutates the test seeds (i.e., the initial messages) to generate new mutated test cases. These new mutated messages will be sent to the IoT device for testing again to continue triggering new state nodes or to verify the stability and security of the device. The purpose of designing this step is to ensure the efficiency in the fuzz testing process and maximize the exploration of all potential states of the device by continuously recording and reusing effective mutation strategies. The benefit of this design is that it not only improves the test coverage rate but also reduces the waste of test time and computing resources by reusing existing effective mutation strategies, thus improving the test efficiency.

[0055] S45: After no new state nodes will appear, perform mutation using unused mutation strategies.

[0056] Specifically, in this step, the fuzz testing continues until no new state nodes are triggered during the testing process. Once all known mutation strategies have been applied and the device no longer generates new responses, the test system automatically switches to unused mutation strategies and performs new mutations on the existing seed messages. These new mutation strategies may include deeper adjustments to specific fields in the message or more complex mutation methods. The purpose of designing this step is to ensure that the fuzz testing covers as many potential states as possible by continuously exploring new mutation strategies until all states of the device are fully tested. The benefit of this design is that it can ensure the comprehensiveness of the test, avoid missing any potential vulnerabilities, and at the same time improve the test efficiency and depth by systematically reusing and expanding mutation strategies, ensuring the stability and security of the device under various extreme conditions.

[0057] As can be seen from the above embodiments, the present application effectively solves the problems of low efficiency of fuzz testing for IoT devices and insufficient representation of the internal state of the device in the prior art by automatically parsing the protocol historical data of the IoT device through a protocol analysis engine based on a large language model, extracting key fields representing the device state, and constructing a state diagram through state nodes. In this way, the testing process can efficiently cover all potential states of the device, improve the coverage rate and accuracy of the fuzz testing, and at the same time accurately reflect the performance of the device in different states, thereby significantly enhancing the security, stability, and robustness of the IoT device.

[0058] Corresponding to the embodiment of the method for fuzz testing stateful Internet of Things devices based on large language models described above, the present application also provides an embodiment of a device for fuzz testing stateful Internet of Things devices based on large language models.

[0059] Figure 2 It is a block diagram of a device for fuzz testing stateful Internet of Things devices based on large language models shown according to an exemplary embodiment. Refer to Figure 2 This device includes:

[0060] Receiving module 1, configured to receive protocol historical data;

[0061] Parsing module 2, configured to parse the protocol historical data using a protocol analysis engine driven by a large prediction model to obtain protocol fields that can represent the system state;

[0062] Normalization module 3, configured to represent the protocol fields in a normalized manner as state nodes;

[0063] Fuzz testing module 4, configured to combine the state nodes into a state graph, and based on this, judge the system states triggered by fuzz testing, and use the untriggered states to guide the fuzz testing.

[0064] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.

[0065] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial descriptions of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0066] Correspondingly, the present application also provides an electronic device, including: one or more processors; a memory, configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method for fuzz testing stateful Internet of Things devices based on large language models as described above.

[0067] Correspondingly, the present application also provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the method for fuzz testing stateful Internet of Things devices based on large language models as described above is implemented.

[0068] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the content disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include well-known knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present application are pointed out by the claims.

[0069] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A fuzzy testing method for stateful IoT devices based on a large language model, characterized in that: include: Receive protocol history data; Using a protocol analysis engine driven by a large prediction model to parse the protocol history data, and obtain a protocol field that can represent the system status; Normalize the protocol fields and represent them as state nodes; The state nodes are combined into a state graph, and the system state triggered by the fuzz test is judged based on the state graph, and the fuzz test is guided to proceed with the untriggered state.

2. The method according to claim 1, characterized in that The protocol historical data is parsed using a protocol analysis engine driven by a large prediction model to obtain protocol fields that can represent the system status, including: According to the IoT device to be tested, install and use the supporting interactive software, interact with the device function by function and intercept the message; Call the big language model open API, send the predefined prompt word template to the big prediction model and get a response; Extract the status field from the large language model response.

3. The method according to claim 2, characterized in that The prompt word template consists of two parts. First, the protocol message is simply input without any prompt words, allowing the large language model to identify the protocol type. Next, the prompt words will require the large language model to extract the fields in the message that can represent the device system status.

4. The method according to claim 2, characterized in that: The status field in the large language model response contains two parts, the fields of the request message and the fields of the response message.

5. The method according to claim 2, characterized in that: The IoT devices interact using protocol messages in JSON format. The fields representing the system status extracted by the large language model are usually the "method" field in the request and the "result" or "params" field in the response. The "method" field represents the function executed by the requesting device. The "method" field and the "result" or "params" field in the response are combined to form a status node, which is used to indicate the current internal state of the device. The "result" or "params" field represents whether the device has successfully executed this function and the related parameters returned.

6. The method according to claim 1, characterized in that The state nodes are combined into a state graph, and the system state triggered by the fuzz test is determined based on the state graph, and the fuzz test is guided to proceed with the untriggered state, including: Collect the normalized state nodes to form a state graph; Start fuzz testing, mutate the message parameters, and collect the responses of IoT devices to the mutated messages; The emergence of a new response means the emergence of a new state node, which is merged into the state diagram; Record the mutation strategy corresponding to the node that triggers the new state, and use the mutation strategy to mutate the seed; Until no new state nodes appear, mutation strategies that have not been used are used for mutation.

7. The method according to claim 6, characterized in that Mutate the parameters of the message, including: There are two types of mutations in message parameters. One is for a single message, including the addition, deletion, and replacement of boundary values ​​and special values ​​of character strings. The other is for message sequences, which shuffles the order of message sequences or copies a message.

8. A stateful IoT device fuzzy testing device based on a large language model, characterized in that: include: A receiving module, used for receiving protocol history data; A parsing module, used for parsing the protocol history data using a protocol analysis engine driven by a large prediction model to obtain a protocol field that can represent a system state; A normalization module, used for normalizing the protocol fields into state nodes; The fuzzy test module is used to combine the state nodes into a state diagram, and judge the system state triggered by the fuzzy test based on the state, and guide the fuzzy test to proceed with the untriggered state.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images