A firmware supply chain vulnerability propagation analysis method based on similarity determination

Abstract

This invention discloses a firmware supply chain vulnerability propagation analysis method based on similarity determination. Based on a collected vulnerability database, third-party software package library, and firmware database, the method extracts features from firmware and third-party software packages and performs similarity determination to identify potential vulnerabilities introduced by third-party software packages in the firmware. This allows for the identification of affected firmware and specific versions, as well as a list of devices containing the vulnerability, thus completing the firmware supply chain vulnerability propagation analysis. This method is highly automated, portable across different firmware file systems, significantly improves the success rate of existing firmware unpacking techniques, accurately identifies third-party software packages referenced in the firmware, greatly reduces the skill requirements for analysts, shortens the time required for traditional firmware analysis, and significantly improves the efficiency of security personnel in firmware supply chain vulnerability propagation analysis.

Description

Technical Field

[0001] This invention relates to the field of firmware supply chain security, specifically, a firmware supply chain vulnerability propagation analysis method based on similarity determination. Background Technology

[0002] In recent years, with the development of computer technology, especially hardware technology, the computing power of devices has become increasingly powerful. Therefore, resource conservation is no longer the primary consideration in firmware development and design. In the current development of IoT devices, developers often introduce third-party software packages due to functional requirements. For example, some small routers use third-party packages as DNS resolution software, and some devices providing web services use Apache or httpd. However, developers often fail to pay attention to the security of these third-party software packages when using them, or they fail to update the versions of these packages in a timely manner to patch vulnerabilities after releasing the firmware. This results in vulnerabilities contained in the third-party packages being introduced into the firmware.

[0003] Vulnerabilities introduced by third-party software packages are often more threatening than those created by developers themselves, as developers frequently lack understanding of the vulnerability's underlying mechanisms and patching methods. However, these vulnerabilities are publicly available, allowing malicious actors to easily understand their principles and exploitation methods, making them more vulnerable and impacting device security. Furthermore, device vendors often withhold the firmware source code after using third-party software, further complicating vulnerability discovery.

[0004] Existing firmware supply chain security analysis technologies mainly suffer from the following problems:

[0005] (1) There are no publicly available large-scale firmware supply chain vulnerability datasets, open-source firmware datasets, and third-party software package datasets that can be directly used for research.

[0006] (2) Existing firmware image disassembly and analysis technology mainly relies on BAT (Binary Analysis Tools) or binwalk tools, which are difficult to deal with firmware with different file systems and have a low unpacking success rate when analyzing large-scale firmware.

[0007] (3) Existing vulnerability discovery methods are mainly divided into: dynamic analysis, such as fuzz-based methods; and static analysis, such as taint analysis-based methods. The above vulnerability discovery work not only relies on a lot of manual operation, but also consumes a lot of time.

[0008] Therefore, a highly usable, fast, and efficient method for analyzing the propagation of firmware supply chain vulnerabilities is needed to address the problem of firmware supply chain security being threatened by vulnerabilities introduced by third-party software packages in the aforementioned firmware. Summary of the Invention

[0009] The purpose of this invention is to address the problem of firmware supply chain security being threatened by vulnerabilities introduced by third-party software packages in firmware, and to propose a firmware supply chain vulnerability propagation analysis method based on similarity determination.

[0010] This invention is achieved through the following technical solution:

[0011] This invention discloses a firmware supply chain vulnerability propagation analysis method based on similarity determination. Its key feature is that, based on a collected vulnerability database, third-party software package library, and firmware database, feature extraction is performed on firmware and third-party software packages, and similarity determination is then conducted to identify potential vulnerabilities introduced by third-party software packages in the firmware. This allows for the identification of affected firmware and specific versions, as well as a list of devices containing the vulnerabilities, thus completing the firmware supply chain vulnerability propagation analysis. The method includes the following steps:

[0012] 1) Data collection and storage; crawl vulnerability data from public vulnerability databases, software package data from third-party software package libraries, and firmware data from firmware databases, and store them in the local vulnerability database, third-party software package database, and firmware database respectively after preprocessing.

[0013] 2) Firmware unpacking and analysis; Based on the firmware data collected in step 1), the improved binwalk tool is used to extract the firmware file system, and the extracted binary executable file is stored for subsequent analysis;

[0014] 3) Firmware feature extraction; Based on the binary executable file obtained in step 2), specific syntax features are extracted as firmware features using an ELF file parsing tool;

[0015] 4) Third-party software package feature extraction; Based on the third-party software package data collected in step 1), the installation path of the binary file is extracted as the third-party software package feature;

[0016] 5) Similarity determination: Based on the firmware features obtained in step 3) and the third-party software package features obtained in step 4), a similarity algorithm is used to calculate the similarity between the firmware and each software package, and finally the third-party software packages referenced in the firmware are obtained.

[0017] 6) Vulnerability propagation analysis; Based on the third-party software packages referenced in the firmware in step 5) and the vulnerability data information in step 1), a relationship mapping table is established. This can identify potential vulnerabilities introduced by third-party software packages in the firmware, thereby identifying the affected firmware and specific versions, as well as a list of devices containing the vulnerability, thus completing the firmware supply chain vulnerability propagation analysis.

[0018] As a further improvement, the vulnerability data in step 1) of this invention covers vulnerability data information from 19 different ecosystems, including: the specific time of vulnerability release, the specific software package name affected by the vulnerability, the specific version range information of the software package, and the patch information corresponding to the vulnerability.

[0019] As a further improvement, the preprocessing operation of the software package data of the third-party software package library in step 1) of the present invention includes extracting different types of compressed packages using different decompression tools, and sequentially obtaining the corresponding verification values ​​of the compressed packages for accuracy verification.

[0020] As a further improvement, the preprocessing operation of the firmware data in the firmware database in step 1) of the present invention includes removing non-firmware files by using a specific character filtering method. The firmware data information includes: device model, firmware version, firmware type, etc.

[0021] As a further improvement, the improved binwalk tool in step 2) of this invention is based on the existing open-source automated parsing tool binwalk, with corresponding parsing plugins written for different file systems, making it applicable to large-scale firmware analysis work.

[0022] As a further improvement, the subsequent analysis in step 2) of this invention includes obtaining version information through dynamic simulation execution using Qemu.

[0023] As a further improvement, the specific syntactic features in step 3) of this invention include visible strings, dependent library files, and referenced function names. Since the compiler adds some fixed visible strings during the compilation and generation of binary executable files, the parser extracts the visible strings of the code segment, data segment, and symbol section of the ELF file as features of the binary file.

[0024] As a further improvement, in step 4) of the present invention, the paths of identical binary executable files in the third-party software package data are basically the same, and after all binary software installation packages are unpacked and installed, their corresponding paths are the installation paths. Therefore, the installation paths of binary files can be extracted as features of the software packages.

[0025] As a further improvement, the similarity algorithm in step 5) of this invention is an improved Jaccard set similarity algorithm, which compares the similarity between the feature set of binary executable files in the firmware file system and the feature set of binary executable files in the software package, and finally obtains the third-party software package referenced in the firmware.

[0026] As a further improvement, the relationship mapping table in step 6) of the present invention is obtained by comparing the versions of the third-party software packages referenced in the firmware obtained in step 5) based on the subsequent analysis in step 2).

[0027] The beneficial effects of this invention are as follows:

[0028] This invention addresses the problem of firmware supply chain security threats posed by vulnerabilities introduced by third-party software packages in firmware. It proposes a firmware supply chain vulnerability propagation analysis method based on similarity determination. This method utilizes collected vulnerability databases, third-party software package libraries, and firmware databases. By extracting features from firmware and third-party software packages and performing similarity determination, it identifies potential vulnerabilities introduced by third-party software packages in the firmware. This allows for the identification of affected firmware and specific versions, as well as a list of devices containing the vulnerabilities, thus completing the firmware supply chain vulnerability propagation analysis.

[0029] The third-party software package feature extraction in step 4) of this invention is based on an ELF file parsing tool. It extracts visible strings in the binary file, as well as dependent library files and referenced library function names, as third-party software package features. This can distinguish different versions of third-party software packages with the same name, significantly improving the efficiency of feature extraction and subsequent similarity determination.

[0030] The similarity determination algorithm in step 5) of this invention is an improved Jaccard set similarity algorithm. Based on the traditional algorithm, it takes into account the weight of elements between sets, assigns higher weights to feature strings with longer lengths, and uses the average of the similarity values ​​of all binary files in the software package to represent the feature value of the software package, which can improve the accuracy of similarity determination.

[0031] The firmware supply chain vulnerability propagation analysis in step 6) of this invention is based on determining the vulnerability information introduced by third-party software packages in the firmware and the firmware version information affected by the vulnerability, and then analyzing the list of devices using the firmware to determine the scope of the vulnerability's propagation impact on the firmware supply chain.

[0032] Compared to existing methods, this method is highly automated, portable across different firmware file systems, significantly improves the success rate of existing firmware unpacking techniques, accurately identifies third-party software packages referenced in the firmware, greatly reduces the skill requirements for analysts, and shortens the time required for traditional firmware analysis. Furthermore, this method can identify affected firmware and specific versions, as well as a list of devices containing the vulnerability, completing firmware supply chain vulnerability propagation analysis and significantly improving the efficiency of security personnel in firmware supply chain vulnerability propagation analysis. Attached Figure Description

[0033] Figure 1 This is a data flow diagram of a firmware supply chain vulnerability propagation analysis method based on similarity determination; Detailed Implementation

[0034] The specific technical solution of the present invention will be further described below with reference to the accompanying drawings:

[0035] Figure 1 This is a flowchart of a firmware supply chain vulnerability propagation analysis method based on similarity determination, specifically including the following steps:

[0036] 1) Firmware disassembly and analysis; The improved binwalk tool is used to disassemble the firmware under test and extract the firmware's file system and binary executable files;

[0037] Specifically, the improvement to the binwalk tool lies in the development of corresponding parsing plugins for different file systems, enabling the tool to automatically identify firmware based on different file systems, thus making it applicable to large-scale firmware analysis work.

[0038] 2) Firmware feature extraction; using a self-developed ELF file parsing tool, the visible strings in the code segment, data segment, and symbol section of the binary executable file are extracted as features of the binary file;

[0039] Specifically, the syntactic features of the firmware binary file include visible strings, dependent library files, and referenced function names. Since the compiler adds some fixed visible strings during the compilation and generation of the binary executable file, the parser extracts the visible strings from the code segment, data segment, and symbol section of the ELF file as features of the binary file.

[0040] 3) Package feature extraction; extract the installation path of binary files in third-party software packages as package features;

[0041] Specifically, the paths of identical binary executable files in the third-party software packages are basically the same, and the corresponding paths of all binary software installation packages after unpacking and installation are the installation paths. Therefore, the installation paths of binary files can be extracted as features of the software packages.

[0042] 4) Similarity determination; Based on the improved Jaccard set similarity algorithm, the similarity value between the features of binary executable files in the firmware file system and the features in the software feature library is calculated, and finally the third-party software packages referenced in the firmware are obtained. The specific steps include:

[0043]

[0044] (a) Analyze each file in the firmware sequentially, extract all binary executable files in the file system, and put them into set P. firm middle;

[0045] (b) For each extracted binary executable file, the `search_in_database` function searches the existing package feature database using the file's path information. Because package conflicts may exist (i.e., different packages may contain the same binary executable file), and the software version cannot be determined solely by the binary file's path information, the function returns a set G containing multiple different package versions. pkg To accelerate the search process, this invention also constructs a reverse mapping table from binary files to software packages during the construction of the software package database.

[0046] (c) For the set G described in (b) pkg For each software package in the set P, analyze whether other binary executables within that package appear in the firmware's binary executables, i.e., determine whether it belongs to set P. firm A subset of the set. If it appears, the version of the package is added to the candidate package set R. pkg and extract the path firm The characteristics of the corresponding binary executable file are stored in the firmware feature set F. firm This is used for subsequent similarity calculations;

[0047] (d) For the candidate software package set R described in (c) pkg Extract the function features f of the candidate software packages in sequence. pkg The set similarity algorithm is used to calculate the package function features f. pkg With firmware feature set F firmThe similarity value s is calculated and compared with the set similarity threshold δ. If the calculated similarity value s exceeds the threshold δ, it can be considered that the firmware references the third-party software package;

[0048] (e) Returns the set of third-party software packages introduced in the firmware and their corresponding version Pkgs.

[0049] 5) Simulated execution; Based on Qemu, the firmware binary file is dynamically simulated and executed, and the software package version information is extracted using specific parameters;

[0050] Specifically, based on step 1), the firmware is disassembled and analyzed to obtain the file system. The executable binary files in the file system are dynamically simulated and executed. Information is extracted using specific parameters such as "-v". Regular expressions are used to match the results of the simulated execution, and finally the software version information in the firmware is determined.

[0051] 6) Version comparison; compare the software reference results in the firmware with the version range information of the software packages affected by the vulnerability in the vulnerability database to identify potential vulnerabilities introduced by third-party software packages in the firmware.

[0052] Specifically, based on the third-party software package information and version information introduced in the firmware as described in steps 4) and 5), and matched with the local vulnerability database, a relationship mapping table is established by comparing the software reference results in the firmware with the version range of software packages affected by vulnerabilities in the vulnerability database, thereby identifying potential vulnerabilities introduced by third-party software packages in the firmware.

[0053] 7) Vulnerability propagation analysis: Based on device information collected from the firmware database, the affected firmware and specific versions, as well as a list of devices containing the vulnerability, can be identified, thereby completing the vulnerability propagation analysis.

[0054] Specifically, based on the potential vulnerability introduced by a third-party software package in the firmware described in step 6), the affected firmware and its specific version information can be determined. Furthermore, by matching this information with device information collected in the local firmware database, a list of real-world devices containing the vulnerability can be obtained, thus completing the vulnerability propagation analysis.

[0055] It will be understood by those skilled in the art that the above description is merely a single example of the invention and is not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A firmware supply chain vulnerability propagation analysis method based on similarity determination, characterized in that, Based on the collected vulnerability databases, third-party software package libraries, and firmware databases, feature extraction is performed on firmware and third-party software packages, and similarity determination is conducted to identify potential vulnerabilities introduced by third-party software packages in the firmware. This allows for the identification of affected firmware and specific versions, as well as a list of devices containing the vulnerability, thus completing the firmware supply chain vulnerability propagation analysis. The analysis includes the following steps: 1) Data collection and storage; crawl vulnerability data from public vulnerability databases, software package data from third-party software package libraries, and firmware data from firmware databases, and store them in the local vulnerability database, third-party software package database, and firmware database respectively after preprocessing. 2) Firmware unpacking and analysis; Based on the firmware data collected in step 1), the improved binwalk tool is used to extract the firmware file system, and the extracted binary executable file is stored for subsequent analysis; 3) Firmware feature extraction; Based on the binary executable file obtained in step 2), specific syntax features are extracted as firmware features using an ELF file parsing tool; 4) Third-party software package feature extraction; Based on the third-party software package data collected in step 1), the installation path of the binary file is extracted as the third-party software package feature; 5) Similarity determination: Based on the firmware features obtained in step 3) and the third-party software package features obtained in step 4), a similarity algorithm is used to calculate the similarity between the firmware and each software package, and finally the third-party software packages referenced in the firmware are obtained. 6) Vulnerability propagation analysis; Based on the third-party software packages referenced in the firmware in step 5) and the vulnerability data information in step 1), a relationship mapping table is established. This can identify potential vulnerabilities introduced by third-party software packages in the firmware, thereby identifying the affected firmware and specific versions, as well as a list of devices containing the vulnerability, thus completing the firmware supply chain vulnerability propagation analysis.

2. The firmware supply chain vulnerability propagation analysis method based on similarity determination according to claim 1, characterized in that, The vulnerability data in step 1) covers vulnerability data information from 19 different ecosystems, including: the specific time of vulnerability release, the specific software package name affected by the vulnerability, the specific version range of the software package, and the corresponding patch information.

3. The firmware supply chain vulnerability propagation analysis method based on similarity determination according to claim 1, characterized in that, The preprocessing operation of the software package data of the third-party software package library in step 1) includes extracting different types of compressed packages using different decompression tools, and obtaining the corresponding verification values ​​of the compressed packages in sequence for accuracy verification.

4. The firmware supply chain vulnerability propagation analysis method based on similarity determination according to claim 1, characterized in that, The preprocessing operation of firmware data in the firmware database in step 1) includes removing non-firmware files by filtering with specific characters. The firmware data information includes: device model, firmware version, firmware type, etc.

5. The firmware supply chain vulnerability propagation analysis method based on similarity determination according to claim 1, 2, 3, or 4, characterized in that, The improved binwalk tool in step 2) is based on the existing open-source automated parsing tool binwalk, with corresponding parsing plugins written for different file systems, making it applicable to large-scale firmware analysis work.

6. The firmware supply chain vulnerability propagation analysis method based on similarity determination according to claim 5, characterized in that, The subsequent analysis in step 2) includes obtaining version information through dynamic simulation execution using Qemu.

7. The firmware supply chain vulnerability propagation analysis method based on similarity determination according to claim 1, 2, 3, 4, or 6, characterized in that, The specific syntactic features in step 3) include visible strings, dependent library files, and referenced function names. Since the compiler adds some fixed visible strings during the compilation and generation of binary executable files, the parser extracts the visible strings of the code segment, data segment, and symbol section of the ELF file as features of the binary file.

8. The firmware supply chain vulnerability propagation analysis method based on similarity determination according to claim 7, characterized in that, In step 4), the paths of identical binary executable files in the third-party software package data are basically the same, and after all binary software installation packages are unpacked and installed, their corresponding paths are the installation paths. Therefore, the installation paths of binary files can be extracted as features of the software packages.

9. The firmware supply chain vulnerability propagation analysis method based on similarity determination according to claim 7, characterized in that, The similarity algorithm in step 5) is an improved Jaccard set similarity algorithm, which compares the similarity between the feature set of binary executable files in the firmware file system and the feature set of binary executable files in the software package, and finally obtains the third-party software package referenced in the firmware.

10. The firmware supply chain vulnerability propagation analysis method based on similarity determination according to claim 1, 2, 3, 4, 6, 8, or 9, characterized in that, The relationship mapping table in step 6) is obtained by comparing the versions of the third-party software packages referenced in the firmware obtained in step 5) based on the subsequent analysis in step 2).

Images