A Datalog-based automated mining method and system for misuse vulnerabilities of third-party components in IoT firmware

Abstract

The present invention discloses a method and system for automatically mining third-party component vulnerabilities in Internet of Things firmware based on Datalog, belonging to the field of Internet of Things security technology. The system includes: a firmware preprocessing module, which collects and preprocesses firmware and stores it in a firmware warehouse; a Datalog encoding module, which extracts files in the firmware and extracts string content, and re-encodes the decompiled results of the binary files using the Datalog encoding method; a third-party component identification and rule extraction module, which identifies third-party components in the firmware based on regular matching and edit distance, and obtains relevant third-party component usage rules; a vulnerability query module, which uses a Datalog query template filled based on the third-party component usage rules to perform logical queries on third-party component misuse vulnerabilities in the firmware. The present invention can quickly and accurately mine third-party component misuse vulnerabilities in Internet of Things firmware.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things security technology, and in particular to a Datalog-based method and system for automatically mining misuse vulnerabilities of third-party components in Internet of Things firmware. Background Art

[0002] With the continuous development of the Internet of Things (IoT), a wide variety of IoT devices have entered millions of homes. These devices are impacting every aspect of our lives. However, the security issues they pose cannot be ignored. In their pursuit of development speed, developers often overlook the security of IoT devices, leading to frequent vulnerabilities. Currently, various botnets prioritize IoT devices, making them a crucial foundation for botnet attacks.

[0003] Because IoT device development has long relied on various third-party components, vulnerabilities caused by these components are becoming a major source of IoT device vulnerabilities. Among these, vulnerabilities arising from the misuse of third-party components are increasing. Third-party components often have cumbersome and complex usage rules. Failure to strictly adhere to these rules can lead to a variety of vulnerabilities, such as memory leaks and privilege escalation attacks. Currently, many systems can detect vulnerabilities caused by the misuse of third-party components at the source code level. However, no system can detect such vulnerabilities at the binary level. Traditional IoT firmware vulnerability detection methods suffer from various flaws. Fuzz testing simulates firmware with a low success rate, cannot detect specific vulnerabilities, and takes a long time to execute. Static analysis cannot be applied on a large scale and requires extensive manual analysis. Symbolic execution suffers from path explosion and a high false positive rate.

[0004] Currently, large-scale automated detection of third-party component misuse in IoT firmware still faces the following challenges: (1) Disassembled code semantic information is missing and logical relationships are unclear. After disassembly, the firmware cannot obtain source code-level semantic information and code context logical relationships, making it difficult to match third-party component usage rules. (2) Low testing efficiency. Current mainstream IoT firmware vulnerability mining methods have defects that limit large-scale application, such as high false positive rates, the need for manual intervention, and low success rates. (3) Third-party component identification is difficult. Unlike the source code level, which can directly obtain the third-party components used through header files, the binary-level firmware does not clearly indicate the third-party components used, and additional design methods are required to identify the third-party components in the firmware.

[0005] Vulnerabilities in IoT firmware caused by the misuse of third-party components can have serious consequences, but there is currently no effective method for detecting such vulnerabilities at the binary level. Restoring the logical relationships in binary code and designing efficient misuse vulnerability query methods are important and necessary for detecting third-party component misuse vulnerabilities in IoT devices. Summary of the Invention

[0006] In response to the shortcomings of mining vulnerabilities caused by misuse of third-party components in IoT firmware, the present invention provides a Datalog-based automated mining method and system for IoT firmware third-party component misuse vulnerabilities. This method can perform query-based vulnerability mining on IoT firmware.

[0007] The specific technical solutions of the present invention are as follows:

[0008] The first object of the present invention is to provide a Datalog-based automated method for mining misuse vulnerabilities of third-party components in IoT firmware, comprising the following steps:

[0009] Step 1: Collect different types of IoT device firmware from different manufacturers, pre-process the firmware to be tested, and build a firmware warehouse;

[0010] Step 2: Extract the string contents from binary and non-binary files in the firmware, decompile the binary files in the firmware, and encode the decompilation results using the Datalog language;

[0011] Step 3: Identify third-party components in the firmware, extract third-party component documentation, and obtain third-party component usage rules;

[0012] Step 4: Fill in the usage rules into the Datalog-based query template, query the vulnerabilities caused by misuse of third-party components in the firmware decompilation results encoded in the Datalog language, and generate a detection report.

[0013] A second object of the present invention is to provide a Datalog-based automated mining system for misuse vulnerabilities of third-party components in IoT firmware, for implementing the above-mentioned method. The mining system comprises:

[0014] The firmware preprocessing module is used to collect various types of firmware from various manufacturers, preprocess the firmware and store it in the firmware warehouse;

[0015] The Datalog encoding module is used to extract binary and non-binary files from the firmware, extract the string content, and re-encode the decompiled results of the binary files using the Datalog encoding method;

[0016] A third-party component identification and rule extraction module, which is used to identify third-party components in the firmware based on regular matching and edit distance, extract third-party component documents, and obtain third-party component usage rules;

[0017] The vulnerability query module is used to fill in the third-party component usage rules into the Datalog-based query template according to the misuse type of the third-party component, and perform logical queries on the vulnerabilities.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] (1) This invention provides an automated mining system for misuse vulnerabilities of third-party components in IoT firmware, and proposes an automated mining technology for misuse vulnerabilities of third-party components in IoT firmware based on Datalog. This solves the problem of efficient mining of misuse vulnerabilities of third-party components in IoT firmware, and can quickly and accurately mine misuse vulnerabilities of third-party components in IoT firmware, which is practical.

[0020] (2) The present invention provides a binary file decompilation and recoding method based on Datalog, which provides an effective method for mining the logical relationship of statements in binary code and effectively restores the inherent logical relationship of the code;

[0021] (3) The present invention provides a third-party component identification method based on regular matching and edit distance, which provides an effective method for identifying third-party components used in firmware and provides a basis for detecting misuse of third-party components;

[0022] (4) The present invention provides a vulnerability query method based on Datalog, which effectively improves the success rate and efficiency of third-party component misuse vulnerability detection and provides an effective basis for large-scale detection of such vulnerabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the overall module structure of the Datalog-based automated mining system for misuse vulnerabilities of third-party components in IoT firmware.

[0024] Figure 2 This is a flowchart of the automated mining method for misuse vulnerabilities of third-party components in IoT firmware based on Datalog;

[0025] Figure 3 This is a schematic diagram of the firmware preprocessing method;

[0026] Figure 4 This is a diagram of the Datalog encoding method;

[0027] Figure 5 Schematic diagram of third-party component identification and rule extraction method;

[0028] Figure 6 Schematic diagram of vulnerability query method. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be noted that the following examples are intended to facilitate understanding of the present invention and do not have any limiting effect on the present invention.

[0030] like Figure 1 As shown, the present invention provides an automated mining system for misuse vulnerabilities of third-party components in IoT firmware based on Datalog, including: a firmware preprocessing module, a Datalog encoding module, a third-party component identification and rule extraction module, and a vulnerability query module.

[0031] The workflow of the entire IoT firmware third-party component misuse vulnerability automated mining system is as follows: Figure 2 As shown, the following steps are included:

[0032] Step 1: Firmware preprocessing: Collect firmware of different types of IoT devices from different manufacturers and preprocess the firmware to be tested. Use binwalk and file magic to filter non-firmware files and build a firmware warehouse.

[0033] Step 2: Datalog encoding: Extract the string content from the binary and non-binary files in the firmware, decompile the binary files in the firmware, and encode the decompiled results using the Datalog language;

[0034] Step 3: Third-party component identification and rule extraction: Analyze the third-party components in the firmware by combining the third-party component names and the string contents of binary and non-binary files in the firmware; collect relevant third-party component documents and analyze the usage rules of the third-party components;

[0035] Step 4: Fill in the usage rules into the query template developed based on the Datalog language, query the vulnerabilities caused by misuse of third-party components in the firmware decompilation results based on Datalog encoding, and generate a detection report.

[0036] In this invention, the core of step 2 is a method for expressing decompilation results based on the Datalog language. Based on experience, the assembly code obtained through traditional decompilation is obscure and difficult to understand, and cannot clearly express the underlying logical relationships within the code. Therefore, in this invention, we propose encoding the decompiled assembly code based on the Datalog language to explore logical relationships, which mainly includes:

[0037] (2-1) Unpack the firmware, obtain the binary files, and extract the string contents in the binary files and non-binary files;

[0038] (2-2) Decompile the binary file and encode the decompiled result into Datalog language.

[0039] In this invention, the core of step 4 is a vulnerability query method based on the Datalog language. Based on experience, traditional fuzz testing methods are unable to specifically find third-party component misuse vulnerabilities, and symbolic execution methods are prone to path explosion problems. Therefore, in this invention, it is proposed to express the disassembly results in a logical language form and perform vulnerability query based on the Datalog language, which mainly includes:

[0040] (4-1) Analyze the possibility of misuse of third-party components based on the usage rules of third-party components;

[0041] (4-2) Based on the analysis of third-party component misuse, write a Datalog-based query template for each type of third-party component misuse;

[0042] (4-3) Fill the usage rules of each third-party component into the Datalog-based query template in order;

[0043] (4-4) Based on the fully filled Datalog query template, according to the time series attributes of the vulnerability, the decompiled result after Datalog language encoding is used to perform vulnerability query using forward or backward traversal to obtain the vulnerabilities caused by the misuse of third-party components and output a detection report; in this embodiment, forward traversal or backward traversal can be selected according to the actual situation.

[0044] The following describes each module separately.

[0045] 1. Firmware preprocessing module

[0046] It is used to execute step 1, collect different types of devices from different manufacturers, and use file type determination technology to complete pre-processing work such as firmware category identification, so as to build a firmware warehouse. The specific process is as follows Figure 3 As shown, the process is as follows:

[0047] Collect different types of IoT device firmware from major manufacturers' homepages;

[0048] Use the binwalk tool and file magic number to determine the file type, filter non-firmware files, and build a firmware repository.

[0049] 2.Datalog encoding module

[0050] It is used to execute step 2, obtain the real firmware from the firmware warehouse, use the firmware unpacking technology to complete the extraction of the binary file content in the firmware, and extract the string content in the firmware binary file and non-binary file, and re-encode the binary decompilation result in the form of Datalog language. The specific process is as follows Figure 4 As shown, the process is as follows:

[0051] Unpack the firmware, obtain the binary files, and extract the string contents in the binary files and non-binary files;

[0052] Decompile the binary file and encode the decompiled result into Datalog language.

[0053] 3. Third-party component identification and rule extraction module is used to execute step 3, using the third-party component name as the matching benchmark, and identifying the third-party components in the firmware based on regular matching and edit distance. Collect the corresponding third-party component documents, and obtain the third-party component usage rules based on manual in-depth analysis. The specific process is as follows Figure 5 As shown, the process is as follows:

[0054] Using the third-party component name as a matching benchmark, regular expression and edit distance matching is performed with the string content extracted from the firmware binary file and non-binary file to obtain the third-party component category used in the firmware;

[0055] Obtain the corresponding third-party component usage documentation based on the identified third-party component category;

[0056] Deeply analyze the usage documentation of third-party components and extract the usage rules of third-party components.

[0057] 4. Vulnerability Query Module

[0058] It is used to execute step 4, analyze the types of third-party component misuse, write a query template based on Datalog for each type of misuse, fill the query template with the third-party component usage rules, and perform a logical query on the third-party component misuse vulnerability. The specific process is as follows: Figure 6 As shown, the process is as follows:

[0059] Analyze the possibility of misuse of third-party components based on the usage rules of third-party components;

[0060] Based on the analysis of third-party component misuse, write a Datalog-based query template for each type of third-party component misuse;

[0061] Fill the usage rules of each third-party component into the Datalog-based query template in order;

[0062] Based on the completed Datalog query template and the time series attributes of the vulnerability, the decompiled results encoded in the Datalog language are selectively traversed to perform vulnerability queries, obtain vulnerabilities caused by the misuse of third-party components, and output a detection report.

[0063] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A Datalog-based automated method for mining misuse vulnerabilities of third-party components in IoT firmware, characterized by: The steps include: Step 1: Collect different types of IoT device firmware from different manufacturers, pre-process the firmware to be tested, and build a firmware warehouse; Step 2: Extract the string contents from binary and non-binary files in the firmware, decompile the binary files in the firmware, and encode the decompilation results using the Datalog language; Step 3: Identify third-party components in the firmware, extract third-party component documentation, and obtain third-party component usage rules; The step 3 is specifically as follows: Step 3.1: Using the third-party component name as a matching benchmark, a matching method based on regular expressions and edit distance is used to match the string content extracted from the firmware binary file and non-binary file to obtain the category of the third-party component used in the firmware; Step 3.2: Obtain the corresponding usage documentation of the third-party component based on the identified third-party component category; Step 3.3: Extract the third-party component usage rules from the third-party component usage documentation; Step 4: Fill in the usage rules into the Datalog-based query template, search for vulnerabilities caused by misuse of third-party components in the firmware decompilation results encoded in the Datalog language, and generate a detection report; The step 4 is specifically as follows: Step 4.1: Based on the third-party component usage rules, enumerate the misuse of third-party components; Step 4.2: Build a Datalog-based query template for each type of third-party component misuse, and fill the query template with the usage rules of each third-party component in order; Step 4.3: Based on the completed Datalog query template, according to the time series attributes of the vulnerability, the decompiled result encoded in the Datalog language is traversed forward or backward to perform vulnerability query, obtain vulnerabilities caused by misuse of third-party components, and output a detection report.

2. The automated mining method for misuse vulnerabilities of third-party components of IoT firmware based on Datalog according to claim 1 is characterized in that: When pre-processing the firmware to be tested, the file type of the firmware to be tested is determined and non-firmware files are filtered out.

3. A Datalog-based automated mining system for misuse vulnerabilities of third-party components in IoT firmware, used to implement the method of claim 1, characterized in that: The excavation system comprises: The firmware preprocessing module is used to collect various types of firmware from various manufacturers, preprocess the firmware and store it in the firmware warehouse; The Datalog encoding module is used to extract binary and non-binary files from the firmware, extract the string content, and re-encode the decompiled results of the binary files using the Datalog encoding method; The third-party component identification and rule extraction module is used to identify third-party components in the firmware based on regular matching and edit distance, extract third-party component documentation, and obtain third-party component usage rules. The module includes: using the third-party component name as a matching benchmark, using a matching method based on regular expressions and edit distance to match the string content extracted from the firmware binary file and non-binary file to obtain the category of the third-party component used in the firmware; obtaining the corresponding third-party component usage documentation based on the identified third-party component category; and extracting the third-party component usage rules from the third-party component usage documentation. The vulnerability query module is used to fill the third-party component usage rules into the Datalog-based query template according to the misuse type of the third-party component, and perform logical queries on the vulnerabilities; including: exhaustively enumerating the misuse of third-party components according to the third-party component usage rules; constructing a Datalog-based query template for each type of third-party component misuse, and filling the usage rules of each third-party component into the query template in sequence; based on the completed Datalog query template, according to the time series attributes of the vulnerability, forward or backward traversal of the decompiled results encoded in the Datalog language is performed to perform vulnerability queries, obtain vulnerabilities caused by the misuse of third-party components, and output a detection report.

Images