A Software Modularization-Based Open Source Vulnerability Function Detection Method for Binary Programs

Abstract

This invention belongs to the field of vulnerability function detection technology, specifically relating to a method for detecting open-source vulnerability functions based on software modularization for binary programs. The method first extracts the directed call graph of binary functions, then uses OSLOM overlapping community detection technology based on statistical significance, and integrates algorithms based on modularity and information theory to cluster the binary functions. Next, based on modularization, feature-based software component analysis is performed. Then, based on the results of the modular software component analysis, the vulnerability types and specific vulnerability functions in open-source components are detected through function signatures, and the corresponding binary vulnerability functions are searched in the binary modules of the reusable components. This method significantly improves detection accuracy and the accuracy of matching individual binary modules compared to component detection based on Louvain modularization and B2SFinder, while also greatly reducing the workload of open-source vulnerability function detection.

Description

Technical Field

[0001] This invention belongs to the field of open-source vulnerability function detection technology, specifically relating to an open-source vulnerability function detection method based on software modularization for binary programs. Background Technology

[0002] Reusing open-source code accelerates program development, and more and more programs and systems incorporate open-source components. However, if an open-source component contains vulnerabilities, the binary program that reuses that component also faces certain risks. For vulnerable functions in open-source components, it's crucial to identify matching functions in the binary program that reuses that component for timely patching. Researchers have proposed various methods for vulnerability detection. In fact, research has found that commercial (or closed-source) programs also contain a large amount of open-source code or components, such as libraries in IoT firmware and the Linux kernel. Therefore, source code-level information can be used for binary program vulnerability detection. However, work closely related to binary vulnerability searching typically involves comparing all functions, leading to an excessively high workload for vulnerability detection.

[0003] Researchers have proposed a series of methods based on binary and source code comparison for detecting vulnerabilities in open-source components. OSSPolice identifies open-source components solely through character type features of strings and exported function names, then checks for vulnerabilities to determine if a reused version of the component exists. B2SFinder uses different weighting algorithms for string, integer, and control flow features to detect open-source component reuse in COTS and analyze potential OSS reuse vulnerabilities. B2SMatcher uses program-level features for coarse matching to identify single-version and multi-version reuse, then uses function-level features for precise matching. This revealed that some applications reused vulnerable versions of open-source components.

[0004] However, the above methods all analyze the entire binary file, failing to pinpoint the location of open-source components. Furthermore, they only identify vulnerabilities within these components without specifying their exact locations. Firstly, analyzing the entire binary file and detecting reused open-source components only confirms their reuse, not the specific module within the binary program. This presents a significant workload for downstream tasks in software supply chain security, as the search scope for vulnerability detection extends to the entire binary code, leading to excessive security analysis effort. Secondly, simply identifying vulnerabilities within the component only concludes that the binary program contains potential vulnerabilities, without accurately identifying specific vulnerable functions within the component or the corresponding binary vulnerability functions. Summary of the Invention

[0005] Current methods for detecting vulnerabilities in open-source components analyze the entire binary file, failing to pinpoint the location of the open-source component. Furthermore, they simply identify vulnerabilities within the component without specifying their exact location, resulting in a large workload and an inability to accurately identify specific vulnerable functions and their defects and problems. This invention provides a method for detecting open-source vulnerability functions based on software modularization for binary programs.

[0006] The solution adopted by this invention to solve its technical problem is: an open-source vulnerability function detection method based on software modularization for binary programs, including software component analysis and open-source vulnerability detection based on program modularization; wherein the software component analysis stage includes program modularization based on directed graphs and software component identification, wherein the program modularization based on directed graphs includes: extracting directed function call information of binary programs; using the directed function call graph as input information for module partitioning, and using the OSLOM overlapping community detection algorithm to cluster the functions in the binary programs;

[0007] Software component identification, on the other hand, extracts the character types and complex branch sequence features of functions from the source code of binary and open-source components, matches each module of the binary program with the open-source components, and identifies the correspondence between binary modules and open-source components.

[0008] During the open-source vulnerability detection phase, function hashing and code normalization techniques are used to create a set of reusable open-source component vulnerability functions through function signatures. For matching binary modules, binary vulnerability functions are detected.

[0009] The aforementioned open-source vulnerability function detection method for binary programs based on software modularization first extracts the directed function call graph of the binary program when modularizing the program based on directed graphs.

[0010] G = (V, E, W)

[0011] E = {(a,b)|a,b∈V}

[0012] W = {W ab |a,b∈V}

[0013]

[0014] In the formula: V is the set of all functions; E represents the set of call edges between all functions, pointing from caller a to callee b; W represents the set of edge weights. ab It is the call weight from function a to function b; n ab E is the number of function calls from a to b. ab It is the set of function call edges from a to b;

[0015] Then, the directed function call graph is used as input information for module partitioning, and the DOFC method based on the OSLOM concept is used to partition the binary program into modules; the partitioning of the binary program includes the following steps:

[0016] (1) Use significance scores to detect important modules, set a convergence threshold, and continue until convergence; the initial node is a single function, calculate the probability of adding adjacent functions to the node, and delete unimportant functions;

[0017] (2) For a set of modules, detect the internal structure of the modules or the merging between modules, and find the smallest module cluster result;

[0018] (3) Detect the hierarchical structure of the module;

[0019] The basic function clustering results are formed, where each module becomes a new node. If there are edges between functions of nodes, then edges are formed between nodes, and the weight of the edge is the sum of the edge weights between functions. This process is repeated to form a new supernetwork, and so on, until the process no longer produces new modules.

[0020] The aforementioned open-source vulnerability function detection method based on software modularization for binary programs compares the binary code with the source code of open-source components during software component identification to determine whether the open-source components are reused. The comparison includes two methods: one is to directly detect the similarity between the binary code and the source code, and the other is to determine the compilation source of the binary program, compile the source code into binary form, and convert it into binary similarity comparison work.

[0021] The aforementioned open-source vulnerability function detection method based on software modularization for binary programs selects character types and complex branch sequence features in functions when directly detecting the similarity between binary code and source code; the character types include strings and their arrays, and exported function names; the complex branch sequence features in functions include if / else and switch / case features.

[0022] The aforementioned open-source vulnerability function detection method based on software modularization for binary programs, for character type features, results in feature matching when the binary module and the feature in the source code are the same; for complex branch sequences in functions, if / else features are matched by the length of the longest common subsequence, and a match is determined when the length of the longest common subsequence exceeds a set threshold. The switch / case in the module is compared with the unordered switch / case list with default branches added in the source code.

[0023] The aforementioned open-source vulnerability function detection method based on software modularization for binary programs includes open-source component vulnerability function detection and binary vulnerability function detection. Open-source component vulnerability detection utilizes a vulnerability signature library to perform vulnerability function detection on OSS. Binary vulnerability detection is based on the binaryAI engine to detect reused binary vulnerability functions in the binary modules that reuse OSS.

[0024] The above-described open-source vulnerability function detection method based on software modularization for binary programs includes the following steps for open-source component vulnerability detection:

[0025] (1) The signature of the OSS to be detected is extracted by function hashing and code normalization. All function hash values ​​of the OSS and the path information of the files to which they belong are extracted and stored in the function hash file.

[0026] (2) Parse all function code lines, remove whitespace and comments in the functions, convert all uppercase characters to lowercase, and store both unnormalized and normalized code line forms in the function signature;

[0027] (3) Compare the target OSS signature with the vulnerability signature database, detect the cloned vulnerability code in OSS, and determine the vulnerable function;

[0028] If the entire code of the target function is included in the vulnerability signature, but not any code in the patch signature, the similarity between the syntax of the target function and the function in the vulnerability signature is calculated using the Jaccard similarity coefficient. If the similarity reaches a threshold, the target function code is considered a clone of the vulnerable code.

[0029]

[0030] The above-described open-source vulnerability function detection method based on software modularity for binary programs includes the following steps for binary vulnerability detection:

[0031] (1) The binaryAI engine creates a set of vulnerable functions that are reused from OSS based on Redis, including function code, source file path, and function features obtained from the feature extraction library;

[0032] (2) By comparing vector similarity, the binary functions that match the set of vulnerability functions in the source code are obtained.

[0033] The aforementioned open-source vulnerability function detection method for binary programs based on software modularity generates a signature library using key technologies such as function verification and core code line extraction. This library includes vulnerability signatures and patch signatures. In the process of generating the signature library, the earliest vulnerability functions and the public code lines of currently publicly disclosed vulnerability functions are first verified to generate an extensible vulnerability signature library. Secondly, key code lines, dependent code lines, and control flow code lines are extracted from the vulnerability functions and patch functions. Finally, vulnerability signatures and patch signatures are generated based on the extracted content.

[0034] The beneficial effects of this invention are as follows: This invention provides an open-source vulnerability function detection method for binary programs based on software modularization. By modularizing binary programs, it improves the efficiency of search tasks in software security analysis. Utilizing the statistical significance-based OSLOM algorithm, a binary module partitioning method DOFC for directed graphs is proposed, which can be extended to directed graphs and provide a global understanding of the program.

[0035] The method of this invention uses Movery's signature library to solve the problem of syntax diversity in vulnerability code cloning; and based on the binaryAI engine, it detects reused binary vulnerability functions in reused OSS binary modules, which only requires matching vulnerability functions in specific modules, greatly narrowing the analysis scope of the binary vulnerability function detection task.

[0036] This invention utilizes the Directed Graph Overlap Clustering (DOFC) method for binary modularization, employs feature comparison technology for modular software component analysis, creates a set of vulnerable functions for open-source components through function signatures, and performs vulnerability function detection on binary program modules that reuse open-source components. Compared to component detection based on Louvain modularization and B2SFinder, BMVul improves detection accuracy by 3.16% and 59.57%, respectively; in terms of accuracy for matching individual binary modules, BMVul improves by approximately 39.43% compared to the Louvain-based method; and in terms of F1-Score, BMVul improves by 8.45% compared to B2SFinder. Module-level detection reduces the search space for vulnerable functions, decreases the workload of open-source vulnerability detection, and is of great significance for software security analysis. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the detection method of the present invention.

[0038] Figure 2 This is a flowchart of the DOFC process of the present invention.

[0039] Figure 3 The TP of the method of this invention and the Louvain-detection method 1m Result comparison chart.

[0040] Figure 4 The results of vulnerability function detection are used to analyze the top ten most frequently reused components in Dataset II. Detailed Implementation

[0041] Before elaborating on the technical solution of this invention, this invention will first briefly introduce the relevant content and research on software modularization and open source vulnerability detection.

[0042] 1. Regarding software modularization

[0043] Software modularization involves clustering software entities, such as classes, modules, and files, grouping entities with similar characteristics or those that collaboratively perform the same function into the same module (cluster). Modules are characterized by high cohesion and low coupling, meaning they are tightly connected internally, facilitating the research and analysis of software structure and functional implementation. Its primary applications are software information recovery, software refactoring, and software component identification.

[0044] Regarding software information recovery, S. Mohammadi et al. proposed a neighborhood tree algorithm using Artifact Dependency Graph (ADG) for clustering. This algorithm creates a tree based on available neighbors in the ADG and clusters the nodes. This method exhibits high stability, outperforming search-based and hierarchical algorithms. ATLBO utilizes a Fuzzy Adaptive Fuzzy Teaching Learning based Optimization (ATLBO) method to solve the software module clustering problem. Based on the current search requirements, this method adaptively selects search operators, thus achieving better performance. J. Sun et al. proposed a probabilistic selection (SPS)-based software module clustering algorithm. This algorithm establishes a network model of the software system structure, transforming software module clustering into graph clustering. Furthermore, it increases the search space through multi-path and iterative operations, demonstrating good clustering performance, convergence speed, and stability.

[0045] Regarding software refactoring, E. Hatami et al. used the Ant Colony Optimization (ACO) algorithm to group dependent modules into the same cluster, achieving better stability and higher convergence. Similarly, BGVarghese R et al. traversed the software structure based on the ACO algorithm and then proposed an extended ACO method for modularizing the software.

[0046] Regarding software component identification, C. Psarras et al. utilized a semantic clustering algorithm optimized based on extracted topic purity scores, reducing the need for developers to assign complex parameters and allowing for easy algorithm tuning. They also employed a post-processing technique to merge extracted topics into classes, thus assisting developers in decision-making. Third-party library reuse has become crucial in modern software development; therefore, LibCUP uses a variant of the standard clustering algorithm DBSCAN, namely ∈-DBSCAN, for automated detection of multi-level library usage patterns.

[0047] 2. Open source vulnerability detection

[0048] Open-source components are core building blocks for developing software systems. Because of their reliance on open-source components, software systems may contain known security vulnerabilities. These weaknesses are often hidden in obvious places, making them easily accessible to attackers. The benefits gained from reusing community-developed libraries may be offset by the costs of timely detection, assessment, and mitigation of these vulnerabilities. The use of open-source components is constantly increasing, as is the number of discovered and publicly disclosed open-source vulnerabilities, primarily in Android applications and commercial software in binary form.

[0049] Brahmastra is an application automation tool that helps app stores and security researchers test third-party components in mobile applications at runtime. Brahmastra is a passive approach that uses dynamic drivers to trigger buggy code given a specific vulnerable version of an open-source component. OSSPolice is a fully automated tool for detecting third-party component clones in Android applications. It quickly analyzes application binaries and novelly introduces a hierarchical indexing scheme to identify potential software license violations and the use of known vulnerable OSS versions. LibScout is a library detection technique that resists common code obfuscation and accurately pinpoints the versions of libraries used in an application. It detects libraries using profiles from a comprehensive library database generated from the original library SDK. The results show that application developers can only slowly adapt to new library versions, exposing their end users to a huge window of vulnerabilities. This approach is the first work to quantify the security impact of third-party libraries on the Android ecosystem.

[0050] B2SMatcher is a fine-grained version identification tool for open-source components in commercial software. It uses program-level features for coarse matching and identification of reuse types, determining two types of reuse relationships: single-version reuse and multi-version reuse. Then, it uses function-level features for precise matching. To extract source code features as accurately as possible, it innovatively uses machine learning methods such as K-means clustering and decision trees to obtain compilation-related source files and extract source code features from them. It extracts features only from compilation-related source files to perform precise version identification, employing abstraction and normalization techniques to eliminate redundant functions across multiple versions. It has discovered that some popular applications, such as Zoom and TeamViewer, reuse vulnerable open-source component versions. FOSSIL is a resilient and efficient new system that combines Bayesian networks with enhanced code obfuscation capabilities, enabling it to discover open-source packages matching those listed in security and reverse engineering reports within malware binaries. FIBER proposes a fine-grained patch presence testing method, elevating similarity-based bug search to a new level: patch presence testing from source to binary scenarios for more accurate bug discovery. Fundamentally, FIBER solves the technical problem of generating binary signatures that accurately represent source code-level patches. It achieves good accuracy and performance, making it highly practical for security analysts. However, FIBER does not account for open-source component variants and may require compiling and searching a large number of binaries.

[0051] Developers often use outdated open-source components and are less aware of potential risks. Therefore, research on vulnerabilities in open-source components can provide practical insights for the sustainable improvement of software systems. Current work on software module clustering primarily focuses on source code clustering, with less attention paid to binary program modularization. BCD uses a fast community detection method, FN, based on a greedy algorithm to partition binary program modules, achieving optimal detection results through a modularity metric. ModX utilizes the Louvain community detection algorithm based on modularity to cluster binary functions. This work relies on the assumption that binary modules do not overlap. However, a function can be called by different functions, leading to situations where a function belongs to multiple modules. Furthermore, the Louvain algorithm is only applicable to undirected networks.

[0052] Based on this, the present invention provides a software modular open-source vulnerability function detection method BMVul for binary programs. The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0053] Example 1: This example describes an open-source vulnerability function detection method for binary programs based on software modularity. Figure 1As shown, the process is divided into two phases: software component analysis based on program modularization and open-source vulnerability detection. In the software component analysis phase, directed function call information from the binary program is first extracted and represented as a graph. Based on the OSLOM overlapping community detection algorithm, functions in the binary program are clustered. Character types and complex branch sequence features of functions in the binary and open-source component source code are extracted. Each module of the binary program is matched with the open-source component to identify the correspondence between binary modules and open-source components, narrowing down the scope of component location. In the open-source vulnerability detection phase, function hashing and code normalization techniques are used to create a set of reusable vulnerable functions for open-source components through function signatures. For the matched binary modules, the vulnerable functions in the binary program are detected. The specific details are as follows.

[0054] I. Software Component Analysis Based on Program Modularization

[0055] 1. A program modularization method based on directed graphs

[0056] Modularization refers to dividing a program into relatively independent components or modules. Each module encapsulates a set of related functions. However, during the compilation of the program source code into a binary program, this modular structure is broken, resulting in all functions being placed in the same binary file. This large number of functions makes binary code analysis difficult. Therefore, this invention proposes a method for modularizing binary code to improve the efficiency of search tasks in software security analysis.

[0057] Based on the limitations of current binary module partitioning work, which focuses on undirected graphs and does not consider overlapping communities, this paper proposes a binary module partitioning method DOFC for directed graphs, based on the OSLOM algorithm utilizing statistical significance. This method is extended to directed graphs and considers the case of overlapping modules. Unlike most binary analysis works that only require local analysis, module partitioning for binary programs is a global understanding of the program. Therefore, the Function Call Graph (FCG) is extracted, which is represented as shown in Equation (1). Here, V is the set of all functions; E represents the set of call edges between all functions, which is represented as shown in Equation (2), pointing from caller a to callee b; W represents the set of edge weights, which is represented as shown in Equation (3). The more times functions are called, the greater the probability that they perform the same function, i.e., belong to the same module.

[0058] G=(V,E,W) (1)

[0059] E={(a,b)|a,b∈V} (2)

[0060] W = {W ab |a,b∈V} (3)

[0061] The number of times a function is called from function a to function b is defined as the directed edge call weight from a to b, and its definition is as follows:

[0062]

[0063] Among them W ab It is the call weight from function a to function b; n ab E is the number of function calls from a to b. ab It is the set of function call edges from a to b.

[0064] Using the directed function call graph as input information for module partitioning, the DOFC method based on the OSLOM concept is used to partition the binary program into modules.

[0065] In the implementation of the DOFC method based on the OSLOM concept, the directed function call graph is used as input to divide the binary program as follows: Figure 2 As shown, it is divided into three stages:

[0066] (1) Use significance scores to detect important modules until convergence. The initial node is a single function. Calculate the probability of adding neighboring functions to the node and delete unimportant functions. A threshold of 0.1 can achieve the best performance. Therefore, the convergence threshold is set to 0.1.

[0067] (2) For a set of modules, detect the internal structure of the modules or the merging between modules, and find the smallest module cluster result.

[0068] (3) Detect the hierarchical structure of the module.

[0069] The above steps form a basic function clustering result, where each module becomes a new node. If there are edges between functions of nodes, then edges are formed between nodes, and the weight of each edge is the sum of the edge weights between the functions. This process then becomes a new supernetwork, and so on, until no more new modules are generated.

[0070] In the implementation of OSLOM, various community detection techniques can be integrated, such as the Infomap heuristic based on random walks, the Copra overlapping community detection method based on label propagation, and the Louvain algorithm based on the concept of modularity. The module outputs of one or more of these algorithms are used as input information for DOFC to perform subsequent module partitioning of the binary program. The more algorithms integrated, the better the final module partitioning effect.

[0071] 2. Software component identification

[0072] Software component identification for binary code involves comparing binary code with the source code of open-source components to determine whether open-source components are reused. Currently, there are two main comparison methods: one is to directly detect the similarity between the binary code and the source code; the other is to determine the compilation source of the binary program (optimization level, architecture, compiler, etc.), compiling the source code into binary form, and then performing binary similarity comparison. However, the latter implementation involves various combinations of compilation configurations, making accurate detection of the compilation source difficult, and the success rate of automatic compilation is low, hindering its practical application.

[0073] Therefore, this invention performs feature-based comparison between binary and source code. Considering that features exist in both binary and source code and are not easily affected by compilation optimization, it selects character type (i.e., strings and their arrays, exported function names) and complex branch sequences (if / else, switch / case) in functions as features.

[0074] In the phase of similarity detection between the binary module and the source code, for character-type features, a feature match is considered achieved when the features in the binary module and the source code are identical. Feature matching is then performed on complex branch sequences of functions based on semantics. The longest common subsequence length is used to match if / else features; a match is determined when the longest common subsequence length exceeds a set threshold. Sequence alignment is then performed between the switch / case statements in the module and the unordered switch / case statements with default branches in the source code. The thresholds for matching each feature are determined empirically. If a feature's matching score exceeds the threshold, the binary module reuses the open-source component.

[0075] II. Open Source Vulnerability Detection

[0076] 1. Detection of vulnerable functions in open-source components

[0077] (1) Source code preprocessing based on standardization

[0078] Most software developers modify code or structure when reusing OSS, and OSS is constantly updated to provide better functionality. However, internal and external changes to OSS can lead to syntactic diversity in vulnerable code. The signature library in Movery can solve this problem of syntactic diversity in cloned vulnerable code. Therefore, this invention utilizes Movery's vulnerability signature library to detect vulnerable functions in OSS.

[0079] The signature library is generated using key technologies such as function verification and core code line extraction, and includes vulnerability signatures and patch signatures. During the signature library generation process, the earliest vulnerable functions and currently publicly disclosed vulnerable functions' public code lines are first verified to generate an extensible vulnerability signature library, addressing the syntactic diversity of vulnerable code caused by internal OSS modifications. Secondly, to address syntactic diversity caused by external OSS modifications, key code lines, dependency code lines, and control flow code lines are extracted from vulnerable and patch functions. Finally, vulnerability signatures and patch signatures are generated based on the extracted content.

[0080] Before performing vulnerability detection on the target OSS, this invention extracts the signature of the OSS to be tested using function hashing and code normalization methods. All function hash values ​​and their associated file path information are extracted from the OSS and stored in a function hash file. Then, all function code lines are parsed, removing whitespace and comments, converting all uppercase characters to lowercase, and storing both unnormalized and normalized code line formats in the function signature.

[0081] (2) Vulnerable function detection

[0082] The target OSS signature is compared with the vulnerability signature database to detect the vulnerability code clone in OSS and identify the vulnerable function. If the entire code of the target function is contained in the vulnerability signature but does not contain any code in the patch signature, as shown in formula (5), the similarity between the syntax of the target function and the function in the vulnerability signature is calculated by the Jaccard similarity coefficient. If it reaches the threshold (0.5), then the target function code is a vulnerability code clone.

[0083]

[0084] 2. Binary vulnerability function detection

[0085] For the detected source code vulnerability functions, the binaryAI engine is used to detect reused binary vulnerability functions within the reusable OSS binary modules. Without dividing the binary modules, it's necessary to compare the source code functions with all binary functions of the program. Based on the results of locating reused OSS binary modules in the above process, vulnerability functions only need to be matched within specific modules, greatly narrowing the analytical scope of the binary vulnerability function detection task.

[0086] The binaryAI engine primarily embeds immediate values, strings, symbols, pseudocode, and control flow graphs of functions, and obtains matching source functions through similarity search. Because this paper detects vulnerable functions reused in binary modules, it does not use the public function set provided by the binaryAI engine for matching. Before function comparison, the engine creates a set of vulnerable functions from the reused OSS based on Redis, including function code, source file paths, and function features obtained from a feature extraction library. Through vector similarity comparison, binary functions matching the source code vulnerability function set are obtained.

[0087] Experimental Example: This experimental example evaluates the detection performance of the BMVul component, matching a unique binary module (TP). 1m The effect is to detect open-source vulnerability functions in the binary for frequently reused component vulnerabilities.

[0088] 1. Dataset

[0089] This experiment uses two datasets to evaluate the detection performance of BMVul.

[0090] Dataset I: The binary programs of reusable components obtained from source code files and the binary programs of known reusable components selected from ISRD and ModX are shown in Table 1. The component source code was obtained from hosting platforms such as Github.

[0091] Table 1 shows the dataset used for evaluating BMVul detection performance.

[0092]

[0093] Dataset II: Collect the top ten most frequently reused components in B2SFinder from hosting platforms such as Github, and obtain the source code of each component for the past three years. The descriptions of the top ten components are shown in Table 2 (the order of the table does not represent the ranking of reuse frequency).

[0094] Table 2 Top 10 Frequently Reused Components

[0095] name type openssl Encryption and Decryption sqlite database libsndfile Audio processing libjpeg-turbo; libpng Image processing freetype; libtiff Font processing unrar; zlib compression expat

[0096] 2. Comparison Methods

[0097] This paper compares the component detection performance of the proposed method BMVul with that of Louvain-detection (based on Louvain module partitioning) and B2SFinder. BMVul is a component detection method based on DOFC module partitioning, which considers the directed properties of the call graph and overlapping module partitioning. It is compared with component detection based on Louvain module partitioning, which does not consider the above properties. B2SFinder is currently the best feature-based binary and open-source component source code comparison method at the file level, and its results are superior to other methods.

[0098] 3. Evaluation Indicators

[0099] The experiment selected Precision and F1-Score as evaluation metrics, defined as shown in the formula.

[0100]

[0101]

[0102]

[0103] 4. Evaluation of component testing results

[0104] Using dataset I, we conducted a component detection evaluation based on software modularity. We compared BMVul, which is based on DOFC module partitioning, with two other methods: Louvain-detection and B2SFinder, which is superior in the work of binary program matching open source component source code. The results are shown in Table 3.

[0105] Table 3 Evaluation of Detection Results

[0106] method P <![CDATA[P lm ]]> F1 score BMVul 75％ 77.8％ 56.5％ Louvain-detection 72.7％ 69.6％ 55.8％ B2SFinder 47％ - 52.1％

[0107] Where P represents the precision with which multiplexed components are identified in the binary module, P 1m This indicates the accuracy of identifying multiplexed components within a single binary module. The table shows that B2SFinder's accuracy is only 47%, Louvain-detection is 72.7%, and BMVul reaches 75%, significantly improving upon these two methods by approximately 3.16% and 59.57%, respectively. The B2SFinder method has a higher number of false positives, leading to a higher false alarm rate. Software-modular detection can reduce the number of false positives, thus lowering the false alarm rate. Analysis of the P-values ​​of Louvain-detection and BMVul... 1mThe change in values ​​shows that BMVul improves upon Louvain-detection by approximately 39.43%. This is because BMVul's software module partitioning differs from Louvain-based algorithms; it uses DOFC (Directed Function Call Graph) for partitioning, considering the directed properties of the function call graph. Furthermore, the Louvain algorithm can only partition a function into one module, neglecting overlapping module partitioning, thus reducing the accuracy of module-based component detection. Therefore, for cases where some functions may belong to multiple modules simultaneously, overlapping detection is used for function clustering, improving the detection accuracy. 1m The F1-Score of P. BMVul component detection reached 56.5%, which is 8.45% higher than B2SFinder and better than current file-level component detection.

[0108] 5. Comparative evaluation of matching unique modules

[0109] A TP (Binary Multiplexed Component, with a detection result of true) that matches a single binary module with a component is represented as a TP. 1m BMVul and Louvain-detection methods target the TP of each binary program. 1m like Figure 3 As shown.

[0110] from Figure 3 As can be seen, BMVul as a whole is above or equal to Louvain-detection, with 5 binary programs in BMVul's TP. 1m Beyond Louvain-detection, the ratio of an exact match to a single binary module to all accurate matches is represented by 1M_Ratio, and its formula is as follows:

[0111]

[0112] The comparison results of 1M_Ratio are shown in Table 4.

[0113] Table 4 1M_Ratio Assessment

[0114]

[0115] The table shows that BMVul's 1M_Ratio is 87.5%, which is about 31.18% higher than Louvain-detection. Therefore, this paper's method for detecting overlapping directed call relationships in programs effectively clusters functions, grouping functions that work together to achieve the same function into the same module, thus improving the accuracy of matching modules with reusable components.

[0116] 6. Open source vulnerability detection for binary modules

[0117] Reusing third-party components may introduce security vulnerabilities into the program. Vulnerable function detection was performed on the top ten most frequently reused components in Dataset II, and the results are as follows: Figure 4 As shown.

[0118] from Figure 4 As can be seen from the source code of the past three years (libtiff source code from 2015-2017), only zlib, libpng, and unrar do not contain vulnerabilities. Freetype and sqlite have a relatively high number of vulnerable functions, reaching 19 and 11 respectively. During software development, if the code of these components containing a large number of vulnerable functions is reused, it will lead to potential security risks for the program. Therefore, it is necessary to detect potential vulnerabilities caused by the reuse of vulnerable components. Vulnerability types and functions were detected in the binary programs of reused vulnerable components in Dataset I. The results of the detected vulnerable functions in the binary modules of reused components are shown in Table 5. Column 5 shows the detection results in this paper; "√" indicates that the vulnerable function is reused in the binary, and "×" indicates that it is not reused.

[0119] Table 5. Open Source Vulnerability Detection Results

[0120]

[0121] As shown in Table 5, based on the results of software component analysis, detecting vulnerable functions in the source code of reusable components' binary modules can uncover reused vulnerable functions in OpenVPN, Lzbench, and Redis-server, going beyond simply detecting potential vulnerabilities and achieving the goal of identifying specific binary vulnerable functions. Furthermore, compared to detecting the entire program file, detecting potential vulnerable functions within the modules of reusable components reduces the number of function matches and avoids comparing and analyzing binary functions from non-reusable components.

[0122] As can be seen from the above, the open-source vulnerability function detection method BMVul proposed in this invention, based on software modularization, matches the source code of open-source components at the binary module level and detects vulnerable functions within the located binary modules. BMVul's component detection accuracy is 59.57% higher than B2SFinder, and its accuracy in matching a single binary module is approximately 39.43% higher than component detection based on Louvain modularization. Module-level detection significantly reduces the search scope for open-source vulnerability functions, which is of great significance for software security analysis.

[0123] The above description is only a preferred embodiment of the present invention and does not limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting open-source vulnerability functions based on software modularization for binary programs, characterized in that: This includes software composition analysis based on program modularization and open-source vulnerability detection; The software component analysis stage includes directed graph-based program modularization and software component identification. The directed graph-based program modularization includes: extracting directed function call information from binary programs; using the directed function call graph as input information for module partitioning; and using the OSLOM-based overlapping community detection algorithm to cluster the functions in the binary programs. Software component identification, on the other hand, extracts the character types and complex branch sequence features of functions from the source code of binary and open-source components, matches each module of the binary program with the open-source components, and identifies the correspondence between binary modules and open-source components. Open source vulnerability detection includes open source component vulnerability function detection and binary vulnerability function detection; among them, open source component vulnerability detection uses vulnerability signature libraries to detect vulnerable functions in OSS, including the following steps: (1) The signature of the OSS to be detected is extracted by function hashing and code normalization. All function hash values ​​of the OSS and the path information of the files to which they belong are extracted and stored in the function hash file. (2) Parse all function code lines, remove whitespace and comments in the functions, convert all uppercase characters to lowercase, and store both unnormalized and normalized code line forms in the function signature; (3) Compare the target OSS signature with the vulnerability signature database, detect the cloned vulnerability code in OSS, and determine the vulnerable function; If the entire code of the target function is included in the vulnerability signature, but not any code in the patch signature, the similarity between the syntax of the target function and the function in the vulnerability signature is calculated using the Jaccard similarity coefficient. If the similarity reaches a threshold, the target function code is considered a clone of the vulnerable code. ； Binary vulnerability detection, based on the binaryAI engine, detects reused binary vulnerability functions in the reused OSS binary module, including the following steps: (1) The binaryAI engine creates a set of vulnerable functions that are reused from OSS based on Redis, including function code, source file path, and function features obtained from feature extraction library; (2) By comparing vector similarity, the binary functions that match the set of vulnerability functions in the source code are obtained.

2. The open-source vulnerability function detection method based on software modularization for binary programs according to claim 1, characterized in that: When modularizing programs based on directed graphs, the first step is to extract the directed function call graph of the binary program. ， ， ， ， In the formula: V It is the set of all functions; E This represents the set of call edges between all functions, pointing to the caller. a To the callee b ; W The set representing edge weights, W ab From function a to function b Call weight; n ab It is from a arrive b The number of function calls; E ab It is from a arrive b The set of function call edges; Then, the directed function call graph is used as input information for module partitioning, and the DOFC method based on the OSLOM concept is used to partition the binary program into modules; the partitioning of the binary program includes the following steps: (1) Use significance scores to detect important modules, set a convergence threshold, and continue until convergence; the initial node is a single function, calculate the probability of adding adjacent functions to the node, and delete unimportant functions; (2) For a set of modules, detect the internal structure of the modules or the merging between modules, and find the smallest module cluster result; (3) Detect the hierarchical structure of the modules; The basic function clustering results are formed, where each module becomes a new node. If there are edges between functions of nodes, then edges are formed between nodes, and the weight of the edge is the sum of the edge weights between functions. This process is repeated to form a new supernetwork, and so on, until the process no longer produces new modules.

3. The open-source vulnerability function detection method based on software modularization for binary programs according to claim 1, characterized in that: In software component identification, binary code is compared with the source code of open-source components to determine whether open-source components are reused. There are two comparison methods: one is to directly detect the similarity between binary code and source code, and the other is to determine the compilation source of the binary program, compile the source code into binary form, and convert it into binary similarity comparison work.

4. The open-source vulnerability function detection method based on software modularization for binary programs according to claim 3, characterized in that: When directly detecting similarity between binary code and source code, select character type and complex branch sequence features in functions; The character types include strings and their arrays, and exported function names; the complex branch sequence characteristics in functions include if / else and switch / case characteristics.

5. The open-source vulnerability function detection method based on software modularization for binary programs according to claim 4, characterized in that: For character-type features, when the binary module has the same feature as the feature in the source code, the result is a feature match; for complex branch sequences in functions, the if / else feature is matched by the length of the longest common subsequence. When the length of the longest common subsequence exceeds a set threshold, it is determined to be a match. The switch / case in the module is compared with the unordered switch / case list with a default branch added in the source code.

6. The open-source vulnerability function detection method based on software modularization for binary programs according to claim 1, characterized in that: The signature library is generated using key technologies such as function verification and core code line extraction, and includes vulnerability signatures and patch signatures. In the process of generating the signature library, the earliest vulnerability functions and the public code lines of currently disclosed vulnerability functions are first verified to generate an extensible vulnerability signature library. Secondly, key code lines, dependent code lines, and control flow code lines are extracted from the vulnerability functions and patch functions. Finally, vulnerability signatures and patch signatures are generated based on the extracted content.

Images