Internet of Things equipment protocol fuzz testing method based on vulnerability knowledge base

Abstract

The invention discloses an Internet of Things equipment protocol fuzz testing method based on a vulnerability knowledge base. According to the method, a test seed is constructed by capturing device traffic, and a big language model is utilized to analyze a public vulnerability knowledge base to guide generation of a test case; designing a variation mechanism combining a manually defined rule and vulnerability knowledge to generate a test case; monitoring whether the internal and external multi-dimensional information detection vulnerabilities of the equipment are triggered or not; and finally, realizing automatic recovery of the equipment state by adopting two strategies of software and hardware, and realizing automation of protocol fuzz testing and vulnerability detection. According to the method, the efficiency and the automation level of closed-source Internet of Things equipment protocol vulnerability mining are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of IoT security testing technology, specifically involving a fuzzy testing method for IoT device protocols based on a vulnerability knowledge base. Background Technology

[0002] With the popularization of IoT technology and the arrival of the era of ubiquitous connectivity, the number of IoT devices deployed in smart homes, industrial IoT, and smart cities has exploded. While providing convenient services and intelligent control, these devices have become important targets for cyberattacks and face serious security threats due to their generally weak security design, lagging firmware updates, and diverse communication protocols. Therefore, efficient and in-depth vulnerability mining of IoT devices has become a key aspect of ensuring their security.

[0003] In IoT device vulnerability discovery, protocol fuzzing, which involves injecting anomalous messages into the target and observing the response, is a commonly used automated technique. However, traditional black-box protocol fuzzing methods for IoT devices with closed-source firmware and invisible state are limited by two main issues: First, test case generation lacks specificity, relying heavily on fixed rules or payload libraries for random / sequential mutations. This makes it difficult to meet complex format verification, authentication, and semantic constraints, resulting in the discarding of numerous invalid messages, low effective exploration rates, and difficulty in uncovering deep logical vulnerabilities. Second, vulnerability awareness and testing process automation are insufficient. Anomalies are often only identified through explicit phenomena such as disconnection or restart, or limited responses, easily missing non-collapsible vulnerability types. Furthermore, the lack of robust automatic recovery and state regression mechanisms hinders continuous testing and closed-loop automation.

[0004] In recent years, Large Language Models (LLMs) have demonstrated powerful capabilities in code understanding, vulnerability pattern recognition, and natural language instruction compliance, providing new ideas for efficient protocol fuzzing. However, effectively integrating them into the closed loop of IoT device protocol fuzzing—building an integrated testing framework capable of understanding protocols, learning vulnerability knowledge, generating high-value test payloads, and possessing multi-dimensional perception and automatic recovery capabilities—remains a pressing technical challenge. Existing research has not yet systematically combined Large Language Models (LLMs) with traditional fuzzing techniques to overcome the aforementioned core bottlenecks in efficiency and perception. Summary of the Invention

[0005] To address the shortcomings of existing black-box IoT device protocol fuzzing methods, such as inefficiency and blind spots in test case generation, limited dimensionality in vulnerability trigger awareness, high false negative rates, and lack of automated device state recovery mechanisms, this invention proposes an IoT device protocol fuzzing method based on a vulnerability knowledge base. This method aims to leverage the semantic understanding and text generation capabilities of a large language model to generate message payloads that are more likely to trigger vulnerabilities. Combined with multi-dimensional monitoring and automated device state recovery mechanisms, it constructs an efficient, accurate, and sustainable automated vulnerability discovery framework for test execution.

[0006] The method for fuzz testing IoT device protocols based on a vulnerability knowledge base includes the following steps:

[0007] Step 1, Initial Seed Message Collection and Structuring: Collect network traffic generated by the target IoT device during configuration management, filter out request message packets used to change device status or configuration (such as POST requests), and filter out general information request messages (such as GET requests). Parse the filtered messages, and parse the message body into a structured set of key-value pairs to form the initial seed message set.

[0008] Step 2, Vulnerability Knowledge Base Construction: Collect publicly available vulnerability information related to the target device or other similar devices from public vulnerability disclosure platforms (including CVE, NVD, CNVD). Utilize a large language model to perform semantic parsing and information extraction on the vulnerability description text, identify and structure the key parameters triggering the vulnerability, related functions, and vulnerability types, thereby constructing a machine-readable and associative vulnerability knowledge base.

[0009] Step 3, Mutation Message Generation: Based on the initial seed message set and the vulnerability knowledge base, mutation request messages are generated collaboratively through the following two methods.

[0010] (1) Batch mutation based on predefined rules: According to the predefined deterministic mutation rules (including key replacement, value replacement, data type obfuscation and boundary value construction), batch mutation operation is performed on key-value pairs in the seed set to quickly generate a large number of test cases with valid formats;

[0011] (2) Targeted mutation based on vulnerability knowledge base: The vulnerability knowledge base is provided as a context to the large language model to guide it to analyze the semantic structure of seed messages for specific vulnerability types and generate targeted mutation message payloads with high trigger probability.

[0012] Step 4, Multi-dimensional Anomaly Monitoring and Judgment: Simultaneously, while sending the generated mutation request packet to the target device, a multi-dimensional monitoring mechanism is initiated on the target device. This includes: connection monitoring to check network connection status; pop-up monitoring to detect web interface anomalies based on automated scripts; server-side monitoring to listen for specific requests initiated by the device to the monitoring server; and SSH monitoring to check internal state changes via SSH when the device supports it. Based on the combined monitoring feedback, a precise determination is made as to whether the vulnerability has been triggered.

[0013] Step 5, Automated Device Status Recovery: While sending mutation messages, establish a mapping relationship between mutation message messages and original message messages. When a device is detected as abnormal but still accessible, send the original message message to restore the configuration. When the device is unresponsive, automatically perform a power-off restart via the smart power socket. If the device still cannot recover after restarting, notify the test personnel via SMS, email, etc., prompting manual intervention.

[0014] The beneficial effects achieved by this invention are as follows:

[0015] (1) Significantly improved testing efficiency and automation level: By integrating the vulnerability knowledge base built by LLM, the generation of test cases has been transformed from "blind random" to "vulnerability knowledge-guided". Combining rule-driven batch mutation and vulnerability knowledge-guided generation modes, the effectiveness and relevance of generated messages have been significantly improved, thereby enabling the generation of more valuable test cases to trigger deep vulnerabilities more quickly.

[0016] (2) Enhanced vulnerability awareness: A multi-dimensional monitoring mechanism integrating connection, interface, external behavior and internal state was designed, which overcomes the limitations of traditional methods that can only detect obvious faults such as device crashes. It can accurately capture potential vulnerabilities such as command injection and cross-site scripting that do not cause the device to crash immediately, and significantly reduces the false negative rate.

[0017] (3) High automation and robustness of the testing process: An innovative automated recovery strategy combining soft recovery based on message mapping and hard recovery based on intelligent power control is introduced. This mechanism can automatically restore the test environment to the maximum extent after the test causes the device to malfunction, reduce manual intervention, ensure the continuity and stability of long-term, large-scale automated fuzz testing, and provide a reliable tool foundation for large-scale security assessment of IoT devices. Attached Figure Description

[0018] Figure 1 This is a flowchart of a specific embodiment of the present invention.

[0019] Figure 2 This is a monitoring function diagram in a specific embodiment of the present invention. Detailed Implementation

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0021] A fuzz testing method for IoT device protocols based on a vulnerability knowledge base, the testing process is as follows: Figure 1 As shown, the core of this method lies in constructing an automated closed loop from real traffic capture to efficient vulnerability discovery. The system first captures device configuration traffic, extracts and structures seed packets; then, it uses publicly disclosed vulnerability information and a Large Language Model (LLM) to build a queryable vulnerability knowledge base; furthermore, it employs a dual-channel mode combining rule-based batch mutation and vulnerability knowledge-guided mutation generation to efficiently generate test cases; during test execution, multi-dimensional monitoring accurately detects device anomalies; finally, through a combination of hardware and software, it achieves automated recovery of device status, ensuring test continuity. The above method mainly includes the following steps:

[0022] Step 1, Data Acquisition and Preprocessing: To obtain high-quality initial test seeds, the test host first connects to the network where the IoT device under test is located via Ethernet or a mirror port, and uses Pyshark or Scapy to capture traffic. During configuration, testers or automated scripts log in to the device's web management interface, traverse all configuration pages, and click buttons such as "Save" and "Apply". The captured HTTP / HTTPS traffic is filtered in real time, retaining only messages with the POST request method that may change the device's state. For each captured request, the system parses its URL, request headers, and message body. If the message body is in application / x-www-form-urlencoded format, it is split into key-value pairs using & and =; if it is in application / json format, a JSON parsing library (such as json.loads) is used to flatten it into a key-value pair structure (e.g., converting {"network": {"ssid": "test"}} to network.ssid: test), forming a structured initial seed set, which is then stored in a database or file.

[0023] Step 2, Vulnerability Knowledge Base Construction: To transform unstructured vulnerability descriptions into exploitable vulnerability knowledge, historical CVE entries for target device models (e.g., TP-Link TL-W***G) and related components (e.g., lighttpd 1.4.5) are first collected via APIs or web crawlers from platforms such as CVE Details and NVD. An existing general-purpose large language model is then requested via API calls, using the description text of each CVE as input. Specific prompts are designed to guide the LLM in performing information extraction tasks, for example: "Please extract from the following vulnerability description: 1) Vulnerability type (command injection / buffer overflow / XSS, etc.); 2) Parameter name that triggers the vulnerability; 3) Possibly involved functions or interfaces; 4) Typical attack payload patterns." The LLM output is parsed and structured, ultimately forming a searchable and associative vulnerability knowledge graph. For example, the following records are generated:

[0024] {Device: “D-Link DCS-5010L”, CVE: “CVE-2017-17020”, Type: “Command Injection”, Trigger Parameter: “system”, Payload Mode: “$(cmd)”}

[0025] Step 3, Mutation Message Generation: This method involves two parallel mutation channels to generate test cases.

[0026] (1) Batch mutation based on predefined rules. This method loads an initial message seed set. The preset mutation rule library includes: key replacement by replacing the original key name with a random string, null value, or excessively long string; injecting strings (e.g., ;reboot) and XSS scripts (e.g., ;reboot) using commands from preset payload libraries such as fuzzdb. <script>alert(1)< / script> Values ​​of strings that are too long (e.g., "A" * 1024) are replaced; numeric values ​​are replaced with strings, negative numbers, or floating-point numbers for data type obfuscation; for numeric parameters, boundary values ​​such as 0, -1, and INT_MAX are generated. For a single seed request, the above rules are applied to multiple (rather than one) key-value pairs at once to generate a batch of mutation requests, greatly improving the efficiency of mutation space exploration.

[0027] (2) Targeted mutation message generation based on vulnerability knowledge base. The current device model being tested, the request message, and the relevant vulnerability patterns retrieved from the vulnerability knowledge base are used as context input to the LLM. The LLM is guided by the following prompt: "Given the request message of target device X, please generate 5 mutation values ​​that can bypass common filters and may trigger the vulnerability based on the parameters that may cause the vulnerability." The LLM then generates a more semantically sophisticated and bypass-capable test payload (such as old_pass$(sleep 5), old_pass\nreboot, etc.) and replaces it in the original request to form a high-value mutation message.

[0028] Step 4, Multi-dimensional Status Monitoring: Through multi-dimensional collaborative monitoring, various anomalies exhibited by IoT devices in fuzz testing are accurately captured, thereby enabling precise identification of vulnerabilities such as command injection, cross-site scripting, and buffer overflows. This method achieves the following: Figure 2 As shown, it is accomplished through the coordinated efforts of four monitoring methods.

[0029] (1) Connection Monitoring: To monitor device connectivity, this monitoring system proactively initiates a short-connection probe to the device's management port after each mutation request is sent. It uses a non-blocking socket implementation with a millisecond timeout (e.g., 300ms). If the connection is successfully established and immediately closed, the device service is considered normal; if the connection times out or is rejected, the service is initially considered abnormal. To eliminate network jitter interference, the system will initiate a secondary verification GET request after a short interval. If there is no response, it is finally confirmed that the device has crashed or the service is unavailable, and the current test case is recorded. This monitoring is a core method for detecting vulnerabilities such as buffer overflows that cause device downtime.

[0030] (2) Pop-up Monitoring: Specifically designed to detect client-side effects of XSS vulnerabilities. This monitoring is based on SeleniumWebDriver driving a headless browser (such as Headless Chrome). When an XSS test request is sent, the monitoring parses the Referer field in the request header, which typically indicates the URL of the configuration page that triggered the request. Subsequently, the automated browser navigates to this URL and attempts to capture the JavaScript pop-up using the driver.switch_to.alert method. If the alert is successfully captured, it indicates that the XSS vulnerability has been successfully triggered and executed on the user's browser, and the relevant test case is recorded and saved to the test log. If the alert is not successfully captured, it waits for a preset timeout (e.g., 5 seconds) and then attempts to capture it again, repeating the attempt a specified number of times (e.g., 2 times); if the pop-up still cannot be captured, it is determined that the test case has not triggered the XSS vulnerability, and the next test case is executed.

[0031] (3)Server - side monitoring: Deploy a dedicated external monitoring server that continuously runs two services: one for listening to ICMP Echo requests (Ping), and the other as a simple HTTP file server hosting an identification file named payload.js. When generating test cases, for command injection - type tests, the instruction ping - c 2 <monitoring server IP> is embedded in the payload; for XSS - type tests, the tag <script src="http: / / <monitoring server IP> / payload.js"> is embedded in the payload. Once a vulnerability is successfully triggered, the device will actively initiate a Ping request to the monitoring server or an HTTP request to download the JS file. The monitoring service captures and logs the source IP and timestamp of these requests in real - time, thus providing evidence of device vulnerability triggering.

[0032] (4)SSH monitoring (for devices supporting SSH debugging): This monitoring provides the most direct verification of internal command execution. When designing the command injection payload, use operations with unique identifiers and no destructiveness, such as: mkdir / tmp / fuzz_mark_{test case ID}. After sending this mutated request, execute the command ls / tmp / fuzz_mark_{test case ID} by establishing an SSH session to check whether the expected directory is created. If the directory exists, it is conclusive evidence that the command injection vulnerability has been exploited and the system command has been executed; if the directory does not exist, it is determined that the test case has not successfully triggered the command injection, and the next test case is continued. This method achieves a closed - loop verification from external input to internal file system state change.

[0033] Step 5, Automatic recovery: To ensure that the fuzz testing process can automatically recover and continue running after an anomaly occurs in the device, when the monitoring detects that the device has a functional anomaly but the network connection is still unblocked, this method will find and resend the corresponding original valid message. This operation aims to roll back the device configuration to the normal state before testing, ensuring that subsequent test cases can continue to execute in the correct configuration baseline environment. When the monitoring confirms that the device is completely unresponsive, this embodiment uses a smart socket to achieve remote power control of the device under test. The recovery method sends a power - on - again instruction to the smart socket by calling the API provided by the smart socket manufacturer. After the device restarts, the system will wait for its services to fully recover and then automatically continue to execute from the breakpoint of the test sequence. If the device still cannot recover to normal after multiple power cycles, an alarm message will be automatically sent to the tester through a preset channel to request manual intervention.

[0034] To verify the feasibility and technical advantages of the method described in this embodiment, a TP-Link TL-WVR900G router was selected as the test target. A total of 25 publicly disclosed CVE vulnerabilities in the TL-WVR series were retrieved. Using the DeepSeek large language model via API calls, the vulnerability type, trigger parameters, interface functions, and exploitation characteristics of each CVE vulnerability description were extracted (null was used if the corresponding element was not mentioned in the description). The results were then stored in a structured JSON format to construct a vulnerability knowledge base. Simultaneously, two mainstream black-box fuzzing tools, Boofuzz ​​and Wfuzz, were selected as comparison objects and tested for 24 hours each. The final number of vulnerabilities discovered was used as the effectiveness evaluation standard.

[0035] The final experimental results show that Boofuzz ​​discovered 6 known CVE vulnerabilities and 2 new command injection issues; Wfuzz discovered 6 new XSS issues; the method described in this application ultimately discovered 22 known CVE vulnerabilities and 13 new issues (7 command injection and 6 XSS), and completely covered the experimental results of Boofuzz ​​and Wfuzz, demonstrating higher testing efficiency and a wider vulnerability coverage.

[0036] Specifically, taking CVE-2017-15613 as an example, this vulnerability describes a command injection issue in the `new-interface` parameter of the `cmxddns.lua` file in TP-Link WVR devices. In actual testing, the method in this embodiment first retrieves the CVE record from the vulnerability knowledge base, obtaining key information such as the vulnerability type "command injection," the triggering parameter "new-interface," and the involved interface "cmxddns." Guided by this, the initial seed message containing the `new-interface` parameter is specifically mutated to generate a test message and send it to the target device. Based on successfully triggering the original vulnerability, this method further discovers that the `new-interface` parameter also exhibits a command injection issue under other interfaces, achieving extended discovery from known vulnerabilities to unknown vulnerabilities, fully demonstrating the knowledge transfer capability and generalization advantage of the method in this application. This case fully demonstrates that the method in this embodiment can not only effectively reproduce known vulnerabilities but also discover new vulnerabilities in similar or related locations based on the guidance of the vulnerability knowledge base, exhibiting superior generalization and discovery capabilities compared to traditional black-box fuzzing tools.

[0037] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. A method for fuzz testing of IoT device protocols based on a vulnerability knowledge base, characterized by: Includes the following steps: Step 1, Initial Seed Collection: Collect network traffic generated by the target IoT device during its configuration and management process, filter out request messages used to change the device status or configuration, parse the filtered request messages to obtain structured key-value pairs, and form an initial seed message set; Step 2, Vulnerability Knowledge Base Construction: Collect publicly available vulnerability information of target IoT devices and similar devices from public vulnerability disclosure platforms; use a large language model to perform semantic parsing and information extraction on vulnerability description text, identify and structure key elements, including vulnerability type and triggering method, and construct a vulnerability knowledge base; Step 3, Mutation Message Generation: Generate mutation messages based on the initial seed message set and vulnerability knowledge base, including generating mutation messages from the initial seed messages based on manually predefined mutation rules, and generating mutation messages from the initial seed messages using a large language model based on the vulnerability knowledge base. Step 4, Abnormal Behavior Monitoring: Send the mutated message to the target IoT device, and determine whether the sent mutated message successfully triggered the device vulnerability by monitoring the device's internal and external multi-dimensional status. Step 5, Automated Equipment Status Recovery: If the equipment is detected to have crashed or become unresponsive due to testing, a power-off restart operation will be performed on the equipment; if the equipment still cannot resume normal service after restarting, an alarm message will be automatically generated and the test personnel will be prompted to manually intervene to restore the equipment status through preset channels.

2. The IoT device protocol fuzzing method based on a vulnerability knowledge base according to claim 1, characterized in that: In step 1, POST request messages used to change device status or configuration are filtered out, and GET request messages used to obtain information are filtered out and removed. When parsing the request messages, if the message body is in key-value pair format, multiple key-value pairs are obtained by separating them with & and the keys and values ​​are extracted by =. If the message body is in JSON format, the JSON is parsed into multiple key-value pairs and the nested fields are flattened into keys with hierarchical paths, represented in key:value format.

3. The IoT device protocol fuzzing method based on a vulnerability knowledge base according to claim 1, characterized in that: In step 2, the security vulnerabilities of relevant IoT devices are retrieved using a public vulnerability disclosure platform; the description text of the vulnerabilities is semantically parsed and information is extracted using a large language model, and the following elements are extracted and structured for storage: vulnerability type, parameter name that triggers the vulnerability, name of the interface or function involved, and corresponding vulnerability exploitation characteristics; the elements are stored in an associated structure of vulnerability type-trigger parameter-interface function-vulnerability exploitation characteristics to build a vulnerability knowledge base.

4. The IoT device protocol fuzzing method based on a vulnerability knowledge base according to claim 1, characterized in that: In step 3, the initial seed message is mutated based on manually predefined mutation rules to generate a mutation request message by performing mutation processing on one or more key-value pairs of the initial seed. The preset mutation rules include key replacement, value replacement, data type obfuscation, and boundary value construction.

5. The IoT device protocol fuzzing method based on a vulnerability knowledge base according to claim 1, characterized in that: In step 3, the generation of mutation messages based on the vulnerability knowledge base is achieved as follows: First, based on the test target URL, relevant vulnerability knowledge is retrieved from the interface function field of the vulnerability knowledge base; then, the initial seed message to be mutated, the retrieved vulnerability knowledge, and the preset mutation generation instructions are combined into a prompt word. The mutation generation instructions are used to guide the large language model to modify or replace the key-value pairs or parameter values ​​in the seed message based on the acquired vulnerability knowledge; finally, the combined prompt word is input into the large language model, which uses its semantic understanding and text generation capabilities to output a mutation message that triggers the target vulnerability type.

6. The IoT device protocol fuzzing method based on a vulnerability knowledge base according to claim 1, characterized in that: In step 4, the multi-dimensional monitoring mechanism for the target IoT device includes: connection monitoring, pop-up monitoring, server-side monitoring, and SSH monitoring. Connection monitoring sends a request to the device to confirm whether the device is responsive when a connection interruption is detected. Pop-up monitoring locates the corresponding page based on the Referer field in the request header and detects abnormal pop-ups through automated access. Server-side monitoring listens for ping requests or file download requests initiated by the device to a remote server to determine whether command injection or XSS has been triggered. SSH monitoring logs into the device via SSH and checks whether the target directory or file created by the injected command exists to determine whether command injection has been triggered.

7. The IoT device protocol fuzzing method based on a vulnerability knowledge base according to claim 1, characterized in that: In step 5, the mapping relationship between the mutated request message and its corresponding original valid request message is saved, and a graded recovery strategy is adopted according to the device anomaly type: when a non-crash anomaly is detected in the device, the original valid request message corresponding to the mutated message is sent to attempt to restore the device configuration to the state before the test; the non-crash anomaly includes, but is not limited to, pop-ups triggered by cross-site script attacks, abnormal external connection requests triggered by command injection, and other situations where the device function is abnormal but the service is not interrupted. When a device crash-type exception is detected, the first step is to attempt to send the original valid request message corresponding to the mutated message for soft recovery. If the device still does not respond after a soft recovery, a power-off restart is performed by sending a request to the smart power outlet to which the device is connected. If the device still cannot resume normal service after a power outage and restart, an alarm message will be sent through a preset notification channel to notify test personnel to manually intervene and restore the device status.

Images