Stateful internet of things device fuzzing method and apparatus based on large language model

Abstract

The application discloses a stateful Internet of Things device fuzzing method and device based on a large language model, which comprises the following steps: receiving protocol history data; using a large prophetic model driven protocol analysis engine to analyze the protocol history data, obtaining protocol fields capable of representing system states; normalizing the protocol fields into state nodes; combining the state nodes into a state graph, and judging the system states triggered by the fuzzing test according to the state graph, and guiding the fuzzing test with the untriggered states.

Description

Technical Field

[0001] This application relates to the field of protocol security testing technology, and in particular to a fuzz testing method and apparatus for stateful Internet of Things devices based on a large language model. Background Technology

[0002] Internet of Things (IoT) devices are intelligent devices connected to the internet that can collect, send, and receive data. They can include home appliances, industrial equipment, wearable devices, sensors, and more. The core purpose of IoT devices is to achieve automation, improve efficiency, and enhance convenience through interconnectivity. However, the diversity and complexity of IoT devices present them with many technical challenges.

[0003] The internal state of IoT devices can be categorized into several aspects, including hardware state, software state, network state, and application state. Hardware state involves the operational status of sensors, the signal strength of communication modules, and the status of power supply and storage. Software state includes firmware version, error logs, and configuration parameters. Network state includes device connectivity, IP address, and data transmission rate. Application state reflects the current task execution status of the device and whether it is responding to user requests. A comprehensive understanding of these internal states is crucial for ensuring the stability and security of IoT devices.

[0004] In current technologies, IoT devices are often exposed to untrusted network environments, facing potential security risks and stability issues. To ensure device security and reliability, fuzzing is widely used as an effective vulnerability discovery technique. By inputting random or anomalous data into the device, fuzzing can help discover problems such as crashes, data leaks, and security vulnerabilities that may occur when the device processes invalid or anomalous input. Fuzzing can not only discover device robustness issues but also reveal firmware vulnerabilities and security problems with network interfaces.

[0005] However, existing fuzzing methods for IoT devices still have some shortcomings. First, due to the wide variety of IoT devices, traditional fuzzing methods often struggle to adapt to the complex protocols and state changes of different devices, resulting in incomplete test coverage. Second, traditional fuzzing methods are insufficient in representing device states, often ignoring the complex behavior of devices under various states, making it difficult to comprehensively assess the security and stability of devices. These problems limit the effectiveness and efficiency of fuzzing. Therefore, how to improve testing efficiency, accurately represent the internal states of devices, and ensure coverage of all potential device states has become an urgent problem to be solved in current technology. Summary of the Invention

[0006] The purpose of this application is to provide a fuzz testing method and apparatus for stateful IoT devices based on a large language model, so as to solve the problems of low testing efficiency and insufficient device state representation in the prior art, thereby improving fuzz testing efficiency and more accurately reflecting the internal system state of the device.

[0007] According to a first aspect of the embodiments of this application, a fuzz testing method for stateful Internet of Things (IoT) devices based on a large language model is provided, comprising:

[0008] Receive historical data of the protocol;

[0009] The protocol historical data is parsed using a protocol analysis engine driven by the Big Oracle model to obtain protocol fields that can represent the system state.

[0010] The protocol fields are normalized and represented as state nodes;

[0011] The state nodes are combined into a state diagram, which is used to determine the system states that have been triggered by the fuzz test, and the fuzz test is guided by the untriggered states.

[0012] According to a second aspect of the embodiments of this application, a fuzz testing apparatus for stateful Internet of Things devices based on a large language model is provided, comprising:

[0013] The receiving module is used to receive historical protocol data;

[0014] The parsing module is used to parse the historical protocol data using a protocol analysis engine driven by the big oracle model to obtain protocol fields that can represent the system state.

[0015] The normalization module is used to normalize the protocol fields into state nodes;

[0016] The fuzzing module is used to combine the state nodes into a state diagram and determine the system states that have been triggered by the fuzzing test, and guide the fuzzing test with the untriggered states.

[0017] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising:

[0018] One or more processors;

[0019] Memory, used to store one or more programs;

[0020] When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0021] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0022] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0023] As demonstrated by the above embodiments, this application employs a protocol analysis engine based on a large language model to parse the historical protocol data of IoT devices and normalizes the extracted protocol fields into state nodes, thereby constructing a state graph. This solves the problems of low efficiency in fuzz testing and insufficient representation of device states in existing technologies. Therefore, it can intelligently generate targeted test inputs, improving the coverage and accuracy of fuzz testing and ensuring a comprehensive representation of the internal system states of the device. Furthermore, it significantly enhances the security and robustness of IoT devices in complex interactive environments, optimizes the testing process, and reduces testing time and resource consumption.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0026] Figure 1 This is a flowchart illustrating a fuzz testing method for stateful Internet of Things (IoT) devices based on a large language model, according to an exemplary embodiment.

[0027] Figure 2 This is a block diagram illustrating a stateful Internet of Things (IoT) device fuzz testing apparatus based on a large language model, according to an exemplary embodiment. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0029] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0031] Figure 1 This is a flowchart illustrating a fuzz testing method for stateful IoT devices based on a large language model, according to an exemplary embodiment. Figure 1 As shown, this method, when applied to a terminal, may include the following steps:

[0032] S1: Receive historical protocol data;

[0033] Specifically, historical protocol data is first collected from interactions between IoT devices and external systems, users, or other devices. This data typically includes the device's request and response messages. By connecting to the device's communication interface (such as Wi-Fi or Bluetooth), the accompanying interactive software or packet capture tools are used to intercept protocol data during device communication. The collected data can be in common formats such as JSON or XML to ensure data integrity and accuracy. This design guarantees that the data used in subsequent analysis is authentic and comprehensive, accurately reflecting the device's behavior and state during actual operation. This provides a reliable basis for subsequent state analysis and fuzz testing, improving test coverage and accuracy.

[0034] S2: The protocol historical data is parsed using a protocol analysis engine driven by the Big Oracle model to obtain protocol fields that can represent the system state; this step includes the following sub-steps:

[0035] S21: Based on the IoT device to be tested, install and use the supporting interactive software, interact with the device function by function and capture the messages;

[0036] Specifically, the first step involves selecting and installing compatible interactive software based on the IoT device under test. This software simulates device operation and communicates with the device. By interacting with each function of the device, the software intercepts corresponding request and response messages, which contain key fields related to the device's status. This method allows for the acquisition of historical protocol data from actual use, ensuring the comprehensiveness and reliability of the data. This design guarantees that the data used in subsequent analysis reflects the actual operation of the device and provides accurate input for large language model-driven protocol analysis, thereby improving the coverage and accuracy of fuzz testing.

[0037] S22: Call the Big Language Model's open API, send the predefined prompt word template to the Big Prophecy Model and receive a response;

[0038] Specifically, the open API of the large language model is invoked, and predefined prompt word templates are sent to the model. These templates are designed according to the device protocol structure, helping the model identify and extract key fields from protocol messages, such as device commands and response results. In this way, the large language model can parse device protocol data and return protocol fields containing device status information, providing a basis for subsequent state diagram construction. This design improves the efficiency and accuracy of protocol parsing, automates the parsing process to reduce manual intervention, and enhances the flexibility and adaptability of the method, ensuring it can handle multiple device protocols.

[0039] The prompt word template consists of two parts. First, the protocol message is input without any prompt words, allowing the large language model to identify the protocol type. Next, the prompt words will require the large language model to extract fields from the message that can represent the device system status.

[0040] S23: Extract the state field from the response of the large language model;

[0041] Specifically, after the large language model returns a response, key information containing the current state of the device is extracted from the response, such as the execution result of the device command and the status of the device function. By parsing the data structure returned by the large language model, fields identifying the device state are identified, such as the "method" field (representing a device function request) and the "result" or "params" field (representing the response result or device parameters). This design ensures accurate capture of the device's state changes during the interaction process, thus providing reliable state nodes for subsequent fuzz testing. In this way, device state extraction can be automated, improving analysis efficiency, reducing manual intervention, and ensuring the accuracy and comprehensiveness of the extracted information.

[0042] The status field in the response of the large language model consists of two parts: fields from the request message and fields from the response message.

[0043] The IoT devices interact using JSON-formatted protocol messages. The fields representing system status extracted by the large language model are typically the "method" field in the request and the "result" or "params" field in the response. The "method" field represents the function requested by the device. Combining the "method" field and the "result" or "params" field in the response forms a status node, which represents the current internal state of the device. The "result" or "params" field represents whether the device successfully executed the function and the relevant parameters returned.

[0044] S3: Normalize the protocol fields to represent them as state nodes;

[0045] Specifically, in this step, the protocol fields extracted from the large language model are normalized and uniformly formatted into state nodes. A state node is an abstract representation of the device's system state at a given moment, including key information such as the device's request commands and response results. For example, the "method" field (representing the function requested by the device) is combined with the "result" or "params" field (representing the execution result or related parameters of the response) into a single state node, thus describing the device's state after a specific interaction. The purpose of this design is to unify fields in different device or protocol formats through normalization, simplifying the subsequent analysis process and ensuring accurate identification and representation of device states. This method improves the accuracy and efficiency of device state identification during fuzz testing, while ensuring compatibility between different devices and reducing potential errors during processing.

[0046] S4: Combine the state nodes into a state diagram, and use this diagram to determine the system states that have been triggered by the fuzz test, and guide the fuzz test with untriggered states; this step includes the following sub-steps:

[0047] S41: Collect the normalized state nodes to form a state diagram;

[0048] Specifically, in this step, the standardized state nodes obtained from the previous steps are combined into a state diagram according to the logic and flow of device interaction. The state diagram represents the transition relationships between different device states by connecting the various state nodes. For example, the transition from a function request state to a response result state can be represented by an edge connecting two state nodes, thus forming the device's state diagram. The purpose of this design is to clearly display the various states of the device and their interrelationships graphically, facilitating the tracking and identification of changes in different states during subsequent fuzzing. By constructing the state diagram, the state changes of the device under different interaction scenarios can be intuitively understood, improving the coverage and effectiveness of fuzzing.

[0049] S42: Start fuzz testing, mutate the parameters of the message, and collect the responses of IoT devices to the mutated message;

[0050] Specifically, the parameters of the messages are mutated, which falls into two categories: one is for individual messages, including adding, deleting, and replacing special boundary values; the other is for message sequences, including reordering the sequence or copying a message. In this step, fuzzing generates new test cases by mutating the parameters of previously collected protocol messages. Mutation methods can include randomly modifying data in the messages, replacing boundary values, adding or deleting fields, etc. Next, the mutated messages are sent to IoT devices, and the devices' responses to these mutated messages are recorded. For example, it might be observed whether the device crashes, returns abnormal data, or exhibits other unexpected behaviors. The purpose of this design is to test the robustness and stability of the device when faced with unforeseen inputs by simulating various abnormal inputs. In this way, potential security vulnerabilities and stability issues of the device can be effectively revealed. The advantage of this design is that it can cover a variety of potential attack scenarios, ensuring that the device can still operate efficiently and securely when faced with abnormal inputs, thereby improving the device's security and robustness.

[0051] S43: The appearance of a new response means that a new state node has appeared. Merge the new state node into the state diagram.

[0052] Specifically, in this step, after the mutated message generated by fuzzing is sent to the IoT device, the device will respond based on the different messages. If the device's response does not match the state node in the existing state diagram, it indicates that the device has entered a new state. At this point, the state field corresponding to the response is extracted to form a new state node, which is then added to the state diagram. This process can continue until no new responses appear. The purpose of this step is to ensure that the test covers all possible device states by dynamically updating the state diagram, avoiding the omission of any potential abnormal states or vulnerabilities. The benefit of this design is that by continuously expanding the state diagram, it can accurately reflect the actual behavior of the device when faced with various inputs, improving the comprehensiveness and effectiveness of the test, while helping developers to more clearly understand the device's state transition process, thereby optimizing the device's security and stability.

[0053] S44: Record the mutation strategy corresponding to triggering the new state node, and use the mutation strategy to mutate the seed;

[0054] Specifically, in this step, whenever a new state node is added to the state graph, the system records the mutation strategy used to trigger that new state node. These mutation strategies include specific mutation methods for message parameters, such as modifying the values ​​of certain fields, adding or deleting fields, changing the order of fields, etc. Next, based on the recorded mutation strategies, the system further mutates the test seed (i.e., the initial message) to generate new mutated test cases. These new mutated messages are then sent back to the IoT device for testing to continue triggering new state nodes or verifying the stability and security of the device. The purpose of this step is to ensure the efficiency of the fuzzing process and maximize the exploration of all potential states of the device by continuously recording and reusing effective mutation strategies. The benefits of this design are that it not only improves test coverage but also reduces the waste of test time and computing resources by reusing existing effective mutation strategies, thereby improving test efficiency.

[0055] S45: Continue mutation until no new state nodes are generated, then use a mutation strategy that has not been used before.

[0056] Specifically, in this step, fuzzing continues until no new state nodes are triggered during the test. Once all known mutation strategies have been applied and the device no longer generates new responses, the testing system automatically switches to unused mutation strategies and applies new mutations to existing seed packets. These new mutation strategies may include deeper adjustments to specific fields in the packets or the use of more complex mutation methods. The purpose of this step is to ensure that fuzzing covers as many potential states as possible by continuously exploring new mutation strategies, until all states of the device are fully tested. The advantage of this design is that it ensures comprehensive testing, avoids missing any potential vulnerabilities, and improves testing efficiency and depth by systematically reusing and expanding mutation strategies, ensuring the stability and security of the device under various extreme conditions.

[0057] As can be seen from the above embodiments, this application automatically parses the protocol history data of IoT devices through a protocol analysis engine based on a large language model, extracts key fields representing device states, and constructs a state graph through state nodes. This effectively solves the problems of low efficiency in fuzz testing of IoT devices and insufficient representation of internal device states in existing technologies. In this way, the testing process can efficiently cover all potential states of the device, improving the coverage and accuracy of fuzz testing, while accurately reflecting the device's performance under different states, thereby significantly improving the security, stability, and robustness of IoT devices.

[0058] Corresponding to the aforementioned embodiments of the fuzz testing method for stateful IoT devices based on large language models, this application also provides embodiments of a fuzz testing apparatus for stateful IoT devices based on large language models.

[0059] Figure 2 This is a block diagram illustrating a fuzzy testing apparatus for stateful Internet of Things (IoT) devices based on a large language model, according to an exemplary embodiment. (Refer to...) Figure 2 The device includes:

[0060] Receiver module 1 is used to receive historical protocol data;

[0061] Parsing module 2 is used to parse the historical protocol data using a protocol analysis engine driven by the big oracle model to obtain protocol fields that can represent the system state.

[0062] Normalization module 3 is used to normalize the protocol fields into state nodes;

[0063] The fuzzing module 4 is used to combine the state nodes into a state diagram and determine the system states that have been triggered by the fuzzing test, and guide the fuzzing test with the untriggered states.

[0064] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0065] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0066] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the fuzz testing method for stateful Internet of Things devices based on a large language model as described above.

[0067] Accordingly, this application also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the fuzz testing method for stateful IoT devices based on a large language model as described above.

[0068] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0069] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A fuzz testing method for stateful Internet of Things (IoT) devices based on a large language model, characterized in that, include: The process involves receiving historical protocol data and parsing it using a protocol analysis engine driven by the Big Prophet model to obtain protocol fields that represent the system state. Specifically, this includes: installing and using the accompanying interactive software on the IoT device to be tested, interacting with the device function by function, collecting historical protocol data, and capturing messages, including corresponding request and response messages containing key fields representing the device state; calling the Big Language Model's open API, sending a predefined prompt word template to the Big Prophet model, and receiving a response; and extracting the state fields from the Big Language Model's response. The prompt word template consists of two parts: first, inputting a protocol message without any prompt words to allow the Big Language Model to identify the protocol type; then, the prompt word will ask the Big Language Model to extract fields from the message that represent the device's system state. The protocol fields are normalized and represented as state nodes; The state nodes are combined into a state diagram, which is used to determine the system states that have been triggered by the fuzz test, and the fuzz test is guided by the untriggered states.

2. The method according to claim 1, characterized in that, The status field in the response of the large language model consists of two parts: fields from the request message and fields from the response message.

3. The method according to claim 1, characterized in that, The IoT devices interact using JSON-formatted protocol messages. The large language model extracts the "method" field in the request and the "result" or "params" field in the response to represent the system state. The "method" field represents the function requested by the device. Combining the "method" field and the "result" or "params" field in the response forms a state node, which represents the current internal state of the device. The "result" or "params" field represents whether the device successfully executed the function and the relevant parameters returned.

4. The method according to claim 1, characterized in that, The state nodes are combined into a state diagram, which is used to determine the system states that have been triggered by the fuzz test. The fuzz test is then guided by the untriggered states, including: The normalized state nodes are collected to form a state diagram; Start fuzz testing, mutate the parameters of the message, and collect the responses of IoT devices to the mutated message; The appearance of a new response means the appearance of a new state node, and the new state node is merged into the state graph; Record the mutation strategy corresponding to triggering the new state node, and use the mutation strategy to mutate the seed; Once no new state nodes are generated, a mutation strategy that has not been used before is employed.

5. The method according to claim 4, characterized in that, The parameters of the message are modified, including: There are two categories of mutations to message parameters. One category is for individual messages, including adding, deleting, and replacing special boundary values ​​with strings. The other category is for message sequences, including reordering the order of message sequences or copying a message.

6. A fuzzy testing device for stateful Internet of Things (IoT) devices based on a large language model, characterized in that, The device is used to perform the fuzz testing method for stateful IoT devices based on a large language model as described in claim 1, and the device comprises: The receiving module is used to receive historical protocol data; The parsing module is used to parse the historical protocol data using a protocol analysis engine driven by the big oracle model to obtain protocol fields that can represent the system state. The normalization module is used to normalize the protocol fields into state nodes; The fuzzing module is used to combine the state nodes into a state diagram and determine the system states that have been triggered by the fuzzing test, and guide the fuzzing test with the untriggered states.

7. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-5.

Images