Iot vulnerability detection method and system based on large language model assisted reasoning

Abstract

The application provides a kind of Internet of Things vulnerability detection method and system based on large language model auxiliary reasoning, comprising: S1, connecting test module and test environment;S2, according to the target of test, propose the security attribute of test;S3, according to the security attribute proposed, the design and synthesis of large language model prompt word are carried out;S4, according to the prompt word obtained in S3, the large language model is called by test module, and the test is autonomously executed under the guidance of large language model, and all input and output in test process are recorded using log;S5, according to the output given by large language model in test process, analyze test result, and verify the authenticity and effectiveness of vulnerability in combination with log in test process.The application reduces the manual labor in Internet of Things vulnerability test work, relies on the logical reasoning ability of large language model, so that the whole detection system can highly autonomously execute vulnerability test work.Solve the problem that automatic test is difficult to detect logical vulnerability.

Description

Technical Field

[0001] This invention provides a method and system for detecting IoT vulnerabilities based on large language model-assisted reasoning, belonging to the field of IoT security technology. Background Technology

[0002] Current IoT vulnerability detection techniques primarily rely on fuzzing testing: this involves inputting random or semi-random data into the target system to discover potential vulnerabilities in its modules. The core idea is to trigger module errors through abnormal input, thereby revealing security vulnerabilities. However, this method is designed for device firmware and can only be used to test firmware vulnerabilities. With technological advancements and evolving customer needs, the desired approach utilizes the reasoning capabilities of Large Language Models (LLMs) to comprehensively test all aspects of an IoT system (such as devices, supporting application modules, and communication protocols), particularly logic-related vulnerabilities. This leverages the unique reasoning capabilities of LLMs, enabling a broader testing scope (both in terms of testable objects and the types of vulnerabilities that can be detected).

[0003] Furthermore, fuzzing tests generally suffer from low efficiency. Summary of the Invention

[0004] This invention provides a method and system for detecting IoT vulnerabilities based on large language model-assisted reasoning. The main purpose is to address the low testing efficiency and inability of other testing methods (such as fuzzing testing) to effectively detect logical vulnerabilities. The technology proposed in this invention can effectively utilize the reasoning capabilities of large language models to understand the connections and operational logic between various components of an IoT system, thereby enabling the detection of logical vulnerabilities.

[0005] Advantage 1: High efficiency, low cost, and minimal human intervention. After providing appropriate prompts, the entire detection system can be automated by leveraging the reasoning capabilities of a large model.

[0006] Advantage 2: It can detect logical flaws. The reasoning ability of large models enables them to exhibit human-like logical thinking ability, thus enabling the automatic detection and identification of logical flaws.

[0007] The specific technical solution is as follows:

[0008] An IoT vulnerability detection system based on large language model-assisted reasoning includes:

[0009] The upper-layer router provides network connectivity for the entire detection system;

[0010] The test host is the core of the entire detection system. A test module runs on the test host, which is responsible for calling the large language model API over the network to utilize the inference capabilities of the large language model. Relying on the inference capabilities of the large language model, the test module running on the test host can autonomously determine the steps to execute the test.

[0011] The testing module connects to various components in the testing environment through different utility classes to acquire information and issue commands. As an intermediary between the testing environment and the large language model, this module is responsible for executing the commands selected by the large language model and returning the execution results and changes in the testing environment to the large language model. The testing module connects to the testing environment through three channels and five utility classes: 1) A firewall control utility class connects to the lower-level router, controlling its firewall behavior to simulate different network states for the testing environment; 2) A traffic analysis utility class connects to the lower-level router to acquire network traffic in the testing environment, providing sufficient information for the large language model to analyze logical vulnerabilities; 3) An ADB utility class connects to the client, providing the large language model with the ability to acquire and control the phone's status; 4) A device control class uses device monitoring and control devices to directly control IoT devices, providing the large language model with the ability to directly control IoT devices; 5) A device monitoring class uses device monitoring and control devices to monitor the real-time status of IoT devices, providing the large language model with the ability to identify the actual status of IoT devices.

[0012] The test environment includes a lower-level router, IoT devices, clients, and device monitoring and control devices. The lower-level router serves as the network control hub, simulating a Wi-Fi router in a user's home environment. This lower-level router connects to the upper-level router, providing network connectivity and a unified simulated network environment for IoT devices and clients. This test environment simulates scenarios where users interact with IoT devices.

[0013] The IoT device is the device under test, simulating the user's IoT device;

[0014] The client is used to run IoT application modules that work with the device under test, simulating a user's mobile phone.

[0015] The steps for running the detection system are as follows:

[0016] An IoT vulnerability detection method based on large language model-assisted reasoning includes the following steps:

[0017] S1. Connect the test module to the test environment. Using the test module running on the test host machine, use the ADB tool to connect the test module to the client in the test environment, use the SSH client to connect the test module to the lower-level router in the test environment, and use the serial interface to connect the test module to the device monitoring and control device.

[0018] S1 uses the following steps to initialize the test environment and test modules, thereby connecting the large language model to the test environment through the test modules:

[0019] S1.1 On the test host machine running the test module, use ADB to connect to the client running the supporting IoT application module, so that the test module uses ADB to obtain the status of the client and send instructions to it;

[0020] S1.2 On the test host machine running the test module, use the SSH client module to connect to the lower-level router in the test environment, so that the test module changes the behavior of the network firewall in the simulated network environment, thereby simulating changes in the network environment;

[0021] S1.3 On the test host machine running the test module, a script module is used to connect to the device monitoring and control device via a serial port, so that the test module can obtain the status of the IoT device under test and change the status of the IoT device under test.

[0022] S2. Determine the security attributes to be tested based on the test objectives. Before starting the test, clarify the test objectives and describe the purpose of the test and its security attributes using natural language;

[0023] S3. Based on the proposed security attributes, design and synthesize the prompt words for the large language model. Based on the test security attributes obtained in S2, generate runtime prompt words using the prompt word template;

[0024] S3 uses the following steps to generate runtime prompts:

[0025] S3.1 Based on the overall testing objective, namely the detection of IoT vulnerabilities, write a test environment description, a large language model utility class description, and a large language model output format description. Combine the above three descriptions directly to generate a preliminary prompt word template.

[0026] S3.2. Based on the actual operating environment of a single test, write the description of the security attributes to be tested and the description of the IoT device under test. Then, directly concatenate the above two parts to generate the component of the runtime prompt.

[0027] S3.3. Concatenate and format the prompt template with the components of the runtime prompt to generate the runtime prompt input to the large language model.

[0028] S4. Based on the prompt words obtained in S3, the test module calls the large language model, executes the test autonomously under the guidance of the large language model, and records all inputs and outputs during the test process using logs.

[0029] S4 employs the following steps:

[0030] S4.1 Implement test utility classes for the large language model API in the test module, so that the large language model API can guide the test module to call different test utility classes as required;

[0031] S4.2 Implement a test utility class in the test module, so that the test utility class can execute test actions and record the output of the test actions, and return the output to the large language model API through the test module;

[0032] S4.3 Implement a logging function in the test module to record all inputs and outputs during test execution.

[0033] S5. Based on the output of the large language model in S4 during the testing process, analyze the test results and verify the authenticity and effectiveness of the vulnerability by combining the logs during the testing process.

[0034] The technical effects of this invention are as follows:

[0035] 1) Reduce manual labor in IoT vulnerability testing by relying on the logical reasoning capabilities of large language models, enabling the entire detection system to perform vulnerability testing work with a high degree of autonomy.

[0036] 2) It solves the problem that automated testing is difficult to detect logical vulnerabilities. The logical reasoning ability of the large language model can understand logical vulnerabilities and can automatically report the logical vulnerabilities that exist in the test environment after the test is completed. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the detection system structure of the present invention;

[0038] Figure 2 This is a schematic diagram of the test module structure and its connection to the test environment of the present invention. Detailed Implementation

[0039] The structure of the detection system provided by this invention is as follows: Figure 1 As shown:

[0040] The entire detection system consists of multiple components. The upper-layer router provides network connectivity for the entire system, while the test host is the core. A test module runs on the test host, which is responsible for calling the large language model API via the network (through the network connection provided by the upper-layer router) to utilize the inference capabilities of the large language model. Relying on the inference capabilities of the large language model, the test module running on the test host can autonomously determine the steps to execute the tests.

[0041] Throughout the testing system, the structure of the test modules running on the host machine and their connection to the test environment are as follows: Figure 2 As shown, the testing module connects to various components in the testing environment through different utility classes to acquire information and issue commands. This testing module acts as an intermediary between the testing environment and the large language model, responsible for executing the commands selected by the large language model and returning the execution results and changes in the testing environment to the large language model. The testing module connects to the testing environment through three channels and five utility classes: 1) A firewall control utility class connects to the lower-level router, which can control the firewall behavior of the lower-level router, thereby simulating different network states for the testing environment; 2) A traffic analysis utility class connects to the lower-level router, thereby acquiring network traffic in the testing environment and providing sufficient information for the large language model to analyze logical vulnerabilities; 3) An ADB utility class connects to the client, thereby providing the large language model with the ability to acquire and control the mobile phone status; 4) A device control class allows direct control of IoT devices using device monitoring and control devices, providing the large language model with the ability to directly control IoT devices; 5) A device monitoring class allows real-time monitoring of IoT devices using device monitoring and control devices, providing the large language model with the ability to identify the actual status of IoT devices.

[0042] like Figure 2 As shown, the test environment consists of a lower-level router, IoT devices (the devices under test, simulating user IoT devices), clients (running IoT application modules that work with the devices under test, simulating user mobile phones), and device monitoring and control devices (cameras, serial port devices, etc.). The lower-level router simulates a Wi-Fi router in a user's home environment, providing network connectivity (the lower-level router connects to the upper-level router to obtain internet access) and a unified simulated network environment (internal network environment) for the IoT devices and clients. This test environment simulates a user's experience using IoT devices.

[0043] The steps for running the detection system are as follows:

[0044] A method for IoT vulnerability detection based on large language model-assisted reasoning. It includes the following steps:

[0045] S1. Connect the test module to the test environment. Using the test module running on the test host machine, use the ADB tool to connect the test module to the client in the test environment, use the SSH client to connect the test module to the network environment control center (i.e., the lower-level router), and use a serial interface (such as USB) to connect the test module to the device monitoring and control device (camera and other devices).

[0046] S1 initializes the test environment and test module using the following steps, thereby connecting the large language model to the test environment through the test module:

[0047] S1.1 On the test host machine running the test module, use ADB to connect to the client running the supporting IoT application module, so that the test module can use ADB to obtain the status of the client and send instructions to it;

[0048] S1.2 On the test host machine running the test module, use the SSH client module to connect to the network control center (lower-level router) in the test environment, so that the test module can change the behavior of the network firewall in the simulated network environment, thereby simulating changes in the network environment;

[0049] S1.3 On the test host machine running the test module, a script module is used to connect to the device monitoring and control device via a serial port, so that the test module can obtain the status of the IoT device under test and change the status of the IoT device under test.

[0050] S2. Determine the security attributes to be tested based on the test objectives (in natural language). Before starting the test, clarify the test objectives and describe the purpose of the test and its security attributes using natural language.

[0051] S3. Based on the proposed security attributes, design and synthesize the prompt words for the large language model. Based on the test security attributes obtained in S2, generate runtime prompt words using the prompt word template;

[0052] S3 uses the following steps to generate runtime prompts:

[0053] S3.1 Based on the overall testing objective, namely the detection of IoT vulnerabilities, write a test environment description, a large language model utility class description, and a large language model output format description. Combine the above three descriptions directly to generate a preliminary prompt word template.

[0054] S3.2. Based on the actual operating environment of a single test, write the description of the security attributes to be tested and the description of the IoT device under test. Then, directly concatenate the above two parts to generate the component of the runtime prompt.

[0055] S3.3. Concatenate and format the prompt template with the components of the runtime prompt to generate the runtime prompt input to the large language model.

[0056] S4. Based on the prompt words obtained in S3, the test module calls the large language model, executes the test autonomously under the guidance of the large language model, and records all inputs and outputs during the test process using logs.

[0057] The following steps are used:

[0058] S4.1 Implement test utility classes for the large language model API in the test module, so that the large language model API can guide the test module to call different test utility classes as required;

[0059] S4.2 Implement a test utility class in the test module so that the test utility class can execute test actions and record the output of the test actions, and return the output to the large language model API through the test module;

[0060] S4.3 Implement a logging function in the test module to record all inputs and outputs during test execution.

[0061] S5. Based on the output of the large language model in S4 during the testing process, analyze the test results and verify the authenticity and effectiveness of the vulnerability by combining the logs during the testing process.

[0062] In practical applications, for example, given security attributes, IoT devices can be controlled through supporting application modules.

[0063] Upon receiving this instruction, the large language model will automatically invoke the utility class in the test module to execute the test. For example:

[0064] 1) The large language model decides to call the "ADB utility class". The test module obtains the actual state of the IoT supporting application module running on the mobile phone through ADB, and then sends this state to the large language model. After receiving the return value, the large language model will know the current state of the application module;

[0065] 2) The Big Prophecy model decides to call the "Device Monitoring Class." The test module obtains the actual status of the IoT device through the environmental monitoring device and sends this status to the Big Prophecy model. After receiving the return value, the Big Prophecy model will know the current actual status of the IoT device.

[0066] 3) The large language model decides to call the "ADB utility class." The test module will use ADB to control the corresponding IoT application module running on the mobile phone, control the corresponding IoT device, and return the control result to the large language model. Thus, the large language model knows that the command to control the IoT device has been successfully executed.

[0067] 4) The large language model decides to call the "Device Monitoring Class". The test module obtains the actual status of the IoT device through the environmental monitoring device and sends this status to the large language model. After receiving the return value, the large language model will know the current actual status of the IoT device.

[0068] By comparing whether the state of the IoT device obtained in 2) and 4) has changed, it can be determined whether the IoT device under test has been successfully controlled by the supporting application module. The large language model can make a judgment on its own to determine whether the attribute "the IoT device can be controlled by the supporting application module" is satisfied.

[0069] If this attribute is not satisfied, a security vulnerability may exist, and the large language model will record the results of this test through the testing module. After all tests are completed, the process and results of this test will be presented in the test report.

Claims

1. An IoT vulnerability detection system based on large language model-assisted reasoning, characterized in that, The detection system includes: The upper-layer router is used to connect the large language model API and the test host machine, as well as to connect the lower-layer router; The test host machine is used to run the test module. The test module calls the large language model API through the upper-layer router, thereby using the reasoning capabilities of the large language model. Relying on the reasoning capabilities of the large language model, the test module can autonomously determine the steps to execute the test. The test environment includes a lower-level router, IoT devices, clients, and device monitoring and control devices. The lower-level router simulates a Wi-Fi router in an IoT user's home environment. The lower-level router connects to the upper-level router, providing Internet connectivity and a unified simulated network environment for IoT devices and clients. This test environment simulates a user's use of IoT devices. The IoT devices are the devices under test. The clients simulate the user's mobile phone and are used to run IoT application modules that are compatible with the devices under test. The testing module is responsible for executing the instructions of the large language model and returning the results of the instruction execution and changes in the testing environment to the large language model. The testing module connects to various components in the testing environment through five utility classes to achieve information acquisition and instruction issuance, specifically including: 1) Connect to the lower-level router through firewall control tools to control the firewall behavior of the lower-level router, thereby simulating different network states for the test environment; 2) Connect to the lower-level router through traffic analysis tools to obtain network traffic in the test environment, providing information for analyzing logical vulnerabilities in the large language model; 3) Connect to the client via ADB utility classes to provide the large language model with the ability to obtain phone status and control the phone; 4) Through the device control class, use device monitoring and control devices to control IoT devices, providing the large language model with the ability to control IoT devices; 5) Through the device monitoring class, the device monitoring and control device is used to monitor the real-time status of IoT devices, providing the large language model with the ability to monitor the real-time status of IoT devices.

2. A method for detecting IoT vulnerabilities based on large language model-assisted reasoning applied to the system as described in claim 1, characterized in that, Includes the following steps: S1. Connecting the test environment and test modules, specifically including: S1.1 On the test host machine running the test module, use the ADB utility class to connect to the client running the supporting IoT application module. The test module uses the ADB utility class to obtain the client's status and send instructions to it. S1.2 On the test host machine running the test module, use an SSH client tool to connect to the lower-level router in the test environment. The test module changes the behavior of the network firewall in the simulated network environment to simulate changes in the network environment. S1.3 On the test host machine running the test module, the script module is used to connect to the device monitoring and control device via serial port. The test module obtains the status of the IoT device under test and changes the status of the IoT device under test. S2. Before starting the test, clarify the test objectives and describe the purpose of the test and its security attributes using natural language; S3. Generate runtime prompts based on the description of the security attributes and the prompt template; S4. Based on the prompt words obtained in S3, the test module calls the large language model, executes the test autonomously under the guidance of the large language model, and records all inputs and outputs during the test process using logs. S5. Based on the output of the large language model in S4 during the testing process, analyze the test results and verify the authenticity and effectiveness of the vulnerability by combining the logs during the testing process.

3. The IoT vulnerability detection method based on large language model-assisted reasoning according to claim 2, characterized in that, S3 uses the following steps to generate runtime prompt words: S3.1 Based on the detection of IoT vulnerabilities, write test environment description, large language model tool class description, and large language model output format description. Combine the above three descriptions directly to generate a preliminary prompt word template. S3.

2. Based on the actual operating environment of a single test, write a description of the security attributes to be tested and a description of the IoT device under test. Combine the two parts directly to generate the components of the runtime prompt. S3.

3. Concatenate and format the components of the prompt word template and the runtime prompt word to generate the runtime prompt word input to the large language model.

4. The IoT vulnerability detection method based on large language model-assisted reasoning according to claim 2, characterized in that, S4 involves the following steps: S4.1 Implement test utility classes for the large language model API in the test module, so that the large language model API can guide the test module to call different test utility classes as required; S4.2 The test module executes test actions and records the test action output through the test utility class, and returns the test action output to the large language model API; S4.3 Implement a logging function in the test module to record all inputs and outputs during test execution.

Images