Firmware vulnerability detection method, device, equipment and medium based on function semantics

Abstract

The present invention provides a firmware vulnerability detection method, device, equipment and medium based on function semantics, which relates to the field of firmware security technology, including: performing firmware parsing processing on the IoT firmware to be detected to extract executable files and their corresponding file information from the IoT firmware to be detected; performing vulnerability function pre-screening processing on the executable files based on the file information to obtain suspicious vulnerability functions contained in the IoT firmware to be detected; performing function similarity matching on suspicious vulnerability functions based on known vulnerability functions contained in a function vulnerability library through a pre-trained target function semantic extraction model to determine whether there are known vulnerability functions with the same source as the suspicious vulnerability functions in the function vulnerability library; if so, determining that the IoT firmware to be detected has vulnerabilities. The present invention can effectively improve the accuracy and robustness of firmware vulnerability detection, and brings significant performance improvement to IoT firmware vulnerability detection.

Description

Technical Field

[0001] The present invention relates to the field of firmware security technology, and in particular to a firmware vulnerability detection method, device, equipment and medium based on function semantics. Background Art

[0002] With the rapid development of the Internet and IoT technology, the era of intelligence is coming. IoT devices have been widely used in various fields, from smart homes to industrial equipment to smart cities. These devices are connected through the Internet, realizing the circulation of data and building a world where everything is connected. But at the same time, the rapid growth of IoT devices also brings security risks. Because these devices are widely distributed, they are vulnerable to various network attacks, posing a serious threat to production safety and user privacy. Therefore, identifying vulnerabilities in IoT devices is of great significance for the security prevention of the industrial Internet. By timely discovering and repairing vulnerabilities in IoT devices, the risk of cyber attacks on the industrial Internet system can be effectively reduced, ensuring the safety and stability of industrial production and operations.

[0003] Since IoT devices are commercial products, most manufacturers will not disclose their source code, and it is indeed difficult to implement the idea of ​​directly auditing the code to discover vulnerabilities. Therefore, researchers usually start from the IoT firmware level, use reverse engineering technology to extract binary code from the firmware, and identify and detect vulnerabilities in the firmware at the binary function level. This method has better applicability and flexibility. With the trend of separation of research and development in the manufacturing process of IoT devices, in order to reduce development costs and improve efficiency, program developers usually introduce third-party open source components. However, the extensive use of third-party components in projects may lead to the problem of vulnerability diffusion. Once a third-party component has a security vulnerability, those projects that reuse the component will also face the same security risks.

[0004] In recent years, many methods have been proposed in the industry for mining known vulnerabilities, but their accuracy and robustness in vulnerability detection for IoT firmware are low, which affects the performance of IoT firmware vulnerability detection. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a firmware vulnerability detection method, device, equipment and medium based on function semantics, which can effectively improve the accuracy and robustness of firmware vulnerability detection and bring significant performance improvement to IoT firmware vulnerability detection.

[0006] In a first aspect, an embodiment of the present invention provides a firmware vulnerability detection method based on function semantics, comprising:

[0007] Performing firmware parsing processing on the IoT firmware to be detected, so as to extract the executable file and its corresponding file information from the IoT firmware to be detected;

[0008] Based on the file information, the executable file is pre-screened for vulnerability functions to obtain suspicious vulnerability functions contained in the IoT firmware to be detected;

[0009] Through the pre-trained target function semantic extraction model, based on the known vulnerability functions contained in the function vulnerability library, the function similarity matching of the suspected vulnerability function is performed to determine whether there is a known vulnerability function with the same source as the suspected vulnerability function in the function vulnerability library;

[0010] If yes, it is determined that the IoT firmware to be tested has a vulnerability.

[0011] In one embodiment, a pre-trained target function semantic extraction model is used to perform function similarity matching on suspicious vulnerability functions based on known vulnerability functions included in a function vulnerability library, including:

[0012] Respectively extracting a first function control flow chart corresponding to a known vulnerability function and a second function control flow chart corresponding to a suspected vulnerability function contained in the function vulnerability library;

[0013] Converting the binary assembly code at each node in the first function control flow graph into an intermediate language, and converting the binary assembly code at each node in the second function control flow graph into an intermediate language;

[0014] Through the pre-trained target function semantic extraction model, function similarity matching is performed on the converted first function control flow graph and the second function control flow graph to obtain the function similarity between the known vulnerability function and the suspected vulnerability function.

[0015] In one embodiment, the objective function semantic extraction model includes two branches, each branch including an intermediate language unit and a graph embedding unit;

[0016] Through the pre-trained target function semantic extraction model, the function similarity matching is performed on the converted first function control flow chart and the second function control flow chart to obtain the function similarity between the known vulnerability function and the suspected vulnerability function, including:

[0017] Generate a first semantic control flow graph based on the converted first functional control flow graph through an intermediate language unit in a branch, and generate a first embedding vector based on the first semantic control flow graph through a graph embedding unit in the branch;

[0018] and, generating a second semantic control flow graph based on the converted second functional control flow graph through an intermediate language unit in another branch, and generating a second embedding vector based on the second semantic control flow graph through a graph embedding unit in the branch;

[0019] The distance between the first embedding vector and the second embedding vector is used as the function similarity between the known vulnerability function and the suspected vulnerability function.

[0020] In one embodiment, the training step of the objective function semantic extraction model includes:

[0021] Obtaining a training data set, the training data set includes binary assembly code for training and its corresponding function control flow chart and intermediate language;

[0022] Combine binary assembly codes with the same name under different architectures in the training data set into positive sample function pairs; and combine binary assembly codes with different names in the training data set into negative sample function pairs;

[0023] The positive sample function pairs and the negative sample function pairs are taken as the function pair data set;

[0024] The initial function semantic extraction model is trained, verified and tested using the function pair dataset to obtain the target function semantic extraction model.

[0025] In one implementation, firmware parsing is performed on the IoT firmware to be detected to extract the executable file and its corresponding file information from the IoT firmware to be detected, including:

[0026] Use reverse engineering tools to decompress the IoT firmware to obtain system files;

[0027] Use the specified command to extract the executable file from the system file and extract the file information corresponding to the executable file.

[0028] In one implementation, the executable file is pre-screened for vulnerability functions based on the file information to obtain suspicious vulnerability functions contained in the IoT firmware to be detected, including:

[0029] Retrieve the file record from the function vulnerability library based on the file information, and determine whether the hash value in the file record is consistent with the hash value corresponding to the executable file;

[0030] If the hash values ​​are consistent, it is determined that the executable file contains the suspicious vulnerability function in the file record;

[0031] If the hash values ​​are inconsistent, the executable file is disassembled to obtain a function set corresponding to the executable file, and it is determined whether the function set corresponding to the executable file contains the suspicious vulnerability function in the file record;

[0032] If the function set corresponding to the executable file does not contain the suspected vulnerability function in the file record, the complement of the function set corresponding to the executable file and the function set in the file record is used as the suspected vulnerability function.

[0033] In one implementation, determining whether there is a known vulnerability function with the same source as the suspected vulnerability function in the function vulnerability library includes:

[0034] If the function similarity between the known vulnerability function and the suspected vulnerability function is greater than a preset similarity threshold, the known vulnerability function is determined to be a known vulnerability function of the same origin as the suspected vulnerability function.

[0035] In a second aspect, an embodiment of the present invention further provides a firmware vulnerability detection device based on function semantics, comprising:

[0036] The firmware parsing module is used to perform firmware parsing processing on the IoT firmware to be detected, so as to extract the executable file and its corresponding file information from the IoT firmware to be detected;

[0037] A vulnerability pre-screening module is used to pre-screen the executable file for vulnerability functions based on the file information to obtain suspicious vulnerability functions contained in the IoT firmware to be detected;

[0038] A similarity matching module is used to perform function similarity matching on suspicious vulnerability functions based on known vulnerability functions contained in the function vulnerability library through a pre-trained function semantic extraction model, so as to determine whether there is a known vulnerability function with the same source as the suspicious vulnerability function in the function vulnerability library;

[0039] The vulnerability detection module is used to determine that the IoT firmware to be detected has a vulnerability when the judgment result of the similarity matching module is yes.

[0040] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.

[0041] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.

[0042] The embodiment of the present invention provides a firmware vulnerability detection method, device, equipment and medium based on function semantics. First, firmware parsing processing is performed on the IoT firmware to be detected to extract the executable file and its corresponding file information from the IoT firmware to be detected; then vulnerability function pre-screening processing is performed on the executable file based on the file information to obtain the suspicious vulnerability function contained in the IoT firmware to be detected; finally, through a pre-trained target function semantic extraction model, function similarity matching is performed on the suspicious vulnerability function based on the known vulnerability function contained in the function vulnerability library to determine whether there is a known vulnerability function with the same source as the suspicious vulnerability function in the function vulnerability library, so as to determine that the IoT firmware to be detected has a vulnerability when the judgment result is yes. The above method obtains the executable file and its corresponding file information by firmware parsing the IoT firmware to be detected, and on this basis, pre-screens the vulnerability function based on the file information to obtain the suspicious vulnerability function in the executable file, and then uses the extraction model based on function semantics and the known vulnerability functions contained in the function vulnerability library to perform function similarity matching on the suspicious vulnerability function, and finally determines whether the IoT firmware to be detected has a vulnerability. The embodiment of the present invention proposes a new function semantic extraction model, which is used to implement firmware vulnerability detection, which can effectively improve the accuracy and robustness of firmware vulnerability detection, and bring significant performance improvement to IoT firmware vulnerability detection.

[0043] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A flowchart of a firmware vulnerability detection method based on function semantics provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a model architecture of an intermediate language unit applicable to a binary function intermediate language provided by an embodiment of the present invention;

[0048] Figure 3 An overall architecture diagram of a BERT pre-training model provided in an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of a function control flow chart provided by an embodiment of the present invention;

[0050] Figure 5 An architecture diagram of an objective function semantic extraction model provided by an embodiment of the present invention;

[0051] Figure 6 A flowchart of building a function vulnerability library provided by an embodiment of the present invention;

[0052] Figure 7 A vulnerability function pre-screening flow chart provided by an embodiment of the present invention;

[0053] Figure 8 An overall flow chart of a firmware vulnerability detection method based on function semantics provided by an embodiment of the present invention;

[0054] Fig. 9 A schematic diagram of the structure of a firmware vulnerability detection device based on function semantics provided by an embodiment of the present invention;

[0055] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] At present, the relevant technologies for mining known vulnerabilities still have the following problems: (1) Since the same source code may be compiled into binary codes of different architectures under different compilers and optimization levels, cross-architecture detection of same-source vulnerabilities is indeed challenging; (2) Traditional binary code analysis methods usually rely on extracting constant strings, instruction types, control flow graphs and other information of the code for numerical quantization, and use the quantized values ​​as the intermediate representation of the binary code. However, these feature extraction methods are often too dependent on expert knowledge, easily missing a large amount of semantic information, and cannot fully express the basic information of the binary code.

[0058] Based on this, the present invention implements a firmware vulnerability detection method, device, equipment and medium based on function semantics, which can effectively improve the accuracy and robustness of firmware vulnerability detection, and bring significant performance improvement to IoT firmware vulnerability detection.

[0059] To facilitate understanding of this embodiment, firstly, a firmware vulnerability detection method based on function semantics disclosed in an embodiment of the present invention is described in detail. Figure 1 The flowchart of a firmware vulnerability detection method based on function semantics is shown, and the method mainly includes the following steps S102 to S108:

[0060] Step S102, performing firmware parsing processing on the IoT firmware to be detected, so as to extract the executable file and its corresponding file information from the IoT firmware to be detected, where the file information is used to describe the file name, architecture, version number, etc. of the executable file.

[0061] In one example, the firmware parsing process mainly includes three functions: firmware decompression, executable file extraction, and component version information extraction. By using the above three functions to process the IoT firmware to be tested, the executable file and its corresponding file information can be obtained.

[0062] Step S104: pre-screen the executable file for vulnerable functions based on the file information to obtain suspicious vulnerable functions contained in the IoT firmware to be detected.

[0063] In one example, it is necessary to pre-build a function vulnerability library containing known vulnerability functions, retrieve file records in the function vulnerability library according to the file information of the executable file, and screen out suspicious vulnerability functions contained in the executable file by comparing the hash value and vulnerability function in the file record with the hash value and function set corresponding to the executable file.

[0064] Step S106, using the pre-trained target function semantic extraction model, based on the known vulnerability functions included in the function vulnerability library, function similarity matching is performed on the suspicious vulnerability function to determine whether there is a known vulnerability function in the function vulnerability library that is homologous to the suspicious vulnerability function.

[0065] Among them, the objective function semantic extraction model is a FSEM (semantic-based binary function similarity recognition model) proposed in an embodiment of the present invention. The model includes two branches, each of which includes an intermediate language unit (FVB, Semantic Extraction Model of Binary Function InterlanguageBased on BERT) and a graph embedding unit (Structure2Vec). In one example, the two branches in the objective function semantic extraction model are used to process the known vulnerability functions and suspected vulnerability functions contained in the function vulnerability library respectively to obtain two corresponding embedding vectors, and the distance between the two embedding vectors is used as the function similarity between the known vulnerability function and the suspected vulnerability function. If the function similarity is greater than the preset similarity threshold, it means that the known function vulnerability and the suspected vulnerability function are of the same origin, that is, the suspected vulnerability function can be identified as a vulnerability function.

[0066] Step S108: If yes, determine that the IoT firmware to be detected has a vulnerability.

[0067] For ease of understanding, the embodiment of the present invention explains the objective function semantic extraction model.

[0068] First, the dataset required for training the objective function semantic extraction model is explained, including the following steps A to C:

[0069] Step A, obtain a training data set, the training data set includes binary assembly code used for training and its corresponding function control flow chart and intermediate language. Step B, combine binary assembly codes with the same name under different architectures in the training data set into positive sample function pairs; and combine binary assembly codes with different names in the training data set into negative sample function pairs. Step C, use the positive sample function pairs and the negative sample function pairs as function pair data sets, and use the function pair data sets to train, verify and test the initial function semantic extraction model to obtain the target function semantic extraction model.

[0070] Open source software is compiled by different manufacturers using different configurations, such as architecture, optimization level, compiler, etc., resulting in great differences in the generation results of binary files. This difference brings great challenges to binary function similarity detection. In order to overcome this problem, the embodiment of the present invention proposes a solution for converting binary assembly code into an intermediate language. By using an intermediate language to represent binary code, the influence caused by different instruction architectures is eliminated, and the intermediate language is converted into a feature vector using a natural language model. This method ensures that the detection of function similarity is more accurate and reliable in a cross-platform or cross-compiler environment. The use of this intermediate language not only reduces the code differences caused by different compilation configurations, but also helps to identify the actual logical and semantic features in the source code, thereby making it easier to detect similar binary functions. In order to further improve the accuracy of binary code similarity detection, the embodiment of the present invention introduces a function control flow graph (CFG) as another important indicator. The function control flow graph can reflect the control flow structure of the function and contains rich semantic information. The embodiment of the present invention uses graph embedding technology to convert the function control flow graph into an embedded vector representation to capture its structural and semantic features. These features can be combined with the aforementioned semantic features based on the intermediate language to form a more comprehensive binary function similarity measure. By integrating the structural semantics of the function control flow graph and the semantic features of the intermediate language, the detection method of the embodiment of the present invention can better overcome the differences caused by different compilation configurations and accurately identify similar binary functions across architectures and compilers. This combination of multi-dimensional features greatly improves the accuracy and robustness of detection, bringing significant performance improvements to firmware vulnerability detection for IoT devices. Through this improvement, the embodiment of the present invention provides a more universal and robust solution for binary code similarity detection, significantly improving the ability to detect similar code under a variety of configurations and architectures.

[0071] Therefore, the embodiment of the present invention first uses a compiler to compile the open source software into different binary files according to different configurations (such as a compilation configuration table shown in Table 1), and then uses a decompilation tool to extract the binary function, and obtains the function control flow chart and intermediate language of the binary function. The embodiment of the present invention defines the binary functions compiled from the same source code through different compiler configurations as the same function. The embodiment of the present invention combines two functions with the same name under different architectures into a positive sample function pair, and randomly selects two functions with different names to form a negative sample function pair. According to this definition, a function pair data set is formed and divided into three non-overlapping data subsets for model training (80%), verification (10%) and testing (10%), as shown in Table 2. The number of binary functions under different compilation configurations. This data set partitioning method helps to comprehensively evaluate the performance and generalization ability of the model.

[0072] Table 1 Compilation configuration table

[0073]

[0074] Table 2 The number of binary functions under different compilation configurations

[0075]

[0076] Based on the above function pair data set, the embodiment of the present invention continues to explain the intermediate language (FVB) unit.

[0077] In order to fully obtain the same semantic features of binary functions under different compilation architectures, the embodiment of the present invention uses a natural language processing model and adopts a siamese architecture to train the entire network end-to-end, and adds a network layer to calculate the cosine distance between two embedded vectors, such as Figure 2 A schematic diagram of a model architecture of an intermediate language unit suitable for a binary function intermediate language is shown, and finally the intermediate language (FVB) unit suitable for a binary function intermediate language required by an embodiment of the present invention is obtained. Among them, BERT (Bidirectional Encoder Representations from Transformers) is a bidirectional encoder representation model based on Transformer, which is widely used in the field of natural language processing. Unlike models that only obtain word semantic information, BERT is more focused on obtaining the comprehensive semantic information of the entire text. It can extract rich text semantic features through unsupervised learning. Compared with traditional unidirectional language models, BERT's bidirectional encoding mechanism enables it to better capture the contextual relationship in the text, thereby showing excellent performance in a variety of NLP (Neuro-Linguistic Programming) tasks. Therefore, the embodiment of the present invention chooses to use BERT as the training model of the function intermediate language, which can quickly and efficiently extract the feature vector of the function. For further information, see Figure 3The overall architecture diagram of a BERT pre-training model shown in the figure includes an input layer, an encoding layer, and an output layer. Among them, W1, W2, W4, W5, and W7 are instructions or parameters after the assembly language is converted into an intermediate language. The input layer of the BERT pre-training model converts the input sentence into a word vector, a text vector, and a position vector. The input sentence starts with the marker CLS, and the two sentences are separated by the marker SEP. Finally, the word vector, the text vector, and the position vector are added as the input of the BERT pre-training model. After the encoding layer (Transform Encoder), the vector representation of each word in the sentence after the semantic information of the whole text is finally output. When two binary functions are given, the embodiment of the present invention adopts the language mask model (MLM) of the BERT pre-training model. During the training process, the standardized instruction of the binary function is replaced with [MASK] with a probability of 15%, so as to force the model to rely on contextual information to predict the standardized instruction, so that the embedded vector of the standardized instruction can more accurately express its semantic information. The two binary functions generate two fixed-dimensional embedded vectors through the BERT pre-training model.

[0078] Based on the above intermediate language (FVB) unit, the embodiment of the present invention continues to explain the objective function semantic extraction model (FSEM).

[0079] The function control flow graph contains rich structural information, such as Figure 4The schematic diagram of a function control flow graph shown in FIG. 1 , wherein each node contains binary assembly code, and the embodiment of the present invention introduces the function control flow graph as another important basis for function similarity. The embodiment of the present invention proposes to generate a graph vector by embedding the function control flow graph into a graph model, and use the cosine distance between the graph vectors to identify function similarity. The structure2vec model is a model for learning graph structure data representation. In Structure2vec, a graph is represented as a structure consisting of nodes and edges. Each node represents an entity in the graph, and each edge represents the relationship between entities. Structure2vec aims to learn the low-dimensional continuous vector representation of nodes, so that similar nodes in the vector space also have similar functions or relationships in the graph structure. This method helps to capture the structural information and correlation between nodes in the vector space. Therefore, the embodiment of the present invention chooses to use node representation learning. Further, the embodiment of the present invention chooses to use Structure2vec as a model for training the function control flow graph. However, Structure2vec has limitations in training the function control flow graph, because the function control flow not only contains the structural information of the function, but also contains intermediate language information in its nodes. Therefore, the embodiment of the present invention proposes a new solution. When the nodes of the function control flow graph are initialized, an intermediate language (FVB) unit is introduced to first generate a feature vector from the intermediate language information of the node, and then the function control flow graph is input into Structure2Vec to generate an embedding vector. The siamese architecture is adopted to train the entire network end-to-end, and a network layer is added to calculate the cosine distance between two embedding vectors, such as Figure 5 The architecture diagram of a target function semantic extraction model shown in FIG. The target function semantic extraction model includes two branches, each of which includes an intermediate language (FVB) unit and a graph embedding (Structure2Vec) unit. Finally, a model that can efficiently extract binary function semantics is generated through training: the target function semantic extraction model (FSEM).

[0080] The process is as follows: input two binary function control flow graphs. First, the basic blocks of the function control flow graphs are converted into feature vectors through the intermediate language (FVB) unit to generate a semantic control flow graph; then, the two semantic control flow graphs are input into the graph embedding (Structure2Vec) unit to generate the function embedding vector: vector and , and then calculate the cosine similarity of the two embedded vectors, the formula is as follows:

[0081] .

[0082] The loss function of the objective function semantic extraction model (FSEM) uses mean-square error (MSE), and the formula is as follows:

[0083] ;

[0084] in, Represents the semantic control flow graph of two binary functions; express and The cosine similarity of the output after passing through the model; express and If the true label and are the same function, then , and For different source functions, ; Indicates the size of a batch in the training set.

[0085] On the basis of the above-mentioned embodiments, the embodiments of the present invention focus on known vulnerability detection, that is, using the characteristics of known vulnerabilities to find possible homologous vulnerabilities in other firmware. Based on this, the purpose of the embodiments of the present invention is to detect open source vulnerabilities in IoT device firmware on a large scale, and propose a binary function similarity detection method based on function semantics, which can also be used in malware analysis, software copyright disputes and other fields. Specifically, the embodiments of the present invention further provide an implementation method of a firmware vulnerability detection method based on function semantics.

[0086] Before executing the aforementioned step S102, it is necessary to build a function vulnerability library in advance. The function vulnerability library is a search library of the firmware vulnerability detection system and is at the basic level in the system. The flowchart of building a function vulnerability library in the embodiment of the present invention is as follows: Figure 6 As shown. Figure 6It can be seen that the sources of function vulnerabilities are divided into two parts. One part is the firmware vulnerabilities announced by various manufacturers. For example, the D-Link router manufacturer announced that multiple devices had the same buffer vulnerability, which was caused by the main() function in htdocs / cgibin. The other part is the vulnerabilities announced by third-party open source components, such as the OBJ_obj2txt function of OpenSSl (before version 1.0.1i), which may cause stack leaks. For vulnerable firmware, the embodiment of the present invention uses a reverse tool (such as Binwalk) to decompress the firmware to obtain the system file; then find the executable file where the vulnerability is located from the system file. For third-party open source components, the embodiment of the present invention uses a compilation tool (Buildroot) to compile the source code into an executable file. For the executable file, first, the embodiment of the present invention extracts its basic information, including the file name, architecture, hash value, version number, etc. Secondly, the embodiment of the present invention needs to extract the vulnerable function in the executable file and extract the function control flow chart. Finally, the extracted vulnerability information is recorded in the function vulnerability library.

[0087] For the aforementioned step S102, an embodiment of the present invention provides an implementation method of performing firmware parsing processing on the IoT firmware to be detected to extract an executable file and its corresponding file information from the IoT firmware to be detected, including: using a reverse tool to decompress the IoT firmware to be detected to obtain a system file; using a specified command to extract the executable file from the system file, and extracting the file information corresponding to the executable file.

[0088] In actual applications, the firmware parsing process mainly includes three sub-functions, namely firmware decompression, executable file extraction, and component version information extraction. In the firmware parsing process: First, the embodiment of the present invention uses the reverse tool Binwalk to decompress the firmware of the IoT device to be detected to obtain the system file. Then, the file command is used to extract the executable file from all files in the system file; finally, the file information of the executable file is extracted, including the file name, architecture, and version number. .

[0089] For the aforementioned step S104, the embodiment of the present invention provides an implementation method of pre-screening the executable file for vulnerability functions based on the file information to obtain the suspicious vulnerability functions contained in the IoT firmware to be detected, see the following steps 1 to 4:

[0090] Step 1: retrieve the file record from the function vulnerability library based on the file information, and determine whether the hash value in the file record is consistent with the hash value corresponding to the executable file;

[0091] Step 2: If the hash values ​​are consistent, it is determined that the executable file contains the suspicious vulnerability function in the file record;

[0092] Step 3: If the Hash values ​​are inconsistent, the executable file is disassembled to obtain a function set corresponding to the executable file, and it is determined whether the function set corresponding to the executable file contains the suspicious vulnerability function in the file record;

[0093] Step 4: If the function set corresponding to the executable file does not contain the suspected vulnerability function in the file record, the complement of the function set corresponding to the executable file and the function set in the file record is used as the suspected vulnerability function.

[0094] Function pre-screening mainly includes function vulnerability library, Hash calculation, disassembly mechanism and obtaining suspicious vulnerability functions. Among them, the function vulnerability library records the detailed information of the vulnerability function (for example, function name, architecture, file, etc.); Hash calculation refers to calculating the Hash value of the executable file; disassembly mechanism refers to obtaining the function of the executable file. For specific implementation, see Figure 7 A vulnerability function pre-screening flow chart is shown. First, an embodiment of the present invention obtains an executable file and its file information; second, according to the file information of the executable file, a file record is retrieved in a function vulnerability library, and a hash value of the executable file is calculated and compared with the hash value in the file record. If the hash values ​​are the same, the executable file contains the vulnerability in the file record and the process ends, otherwise the process continues to execute; thirdly, IDA PRO is used to disassemble the executable file to obtain its function set. If the function set contains the recorded vulnerability function, it is marked as a suspicious vulnerability function, otherwise the process continues to execute; from then on, considering that the executable file does not correctly identify the function name during disassembly, the embodiment of the present invention uses the complement of the function set in the file record (including functions without vulnerabilities) and the executable file function set as a difference function, and marks the difference function as a suspicious vulnerability function; finally, the embodiment of the present invention inputs the suspicious vulnerability function into the target function semantic extraction model.

[0095] For the aforementioned step S106, the embodiment of the present invention further provides an implementation method of performing function similarity matching on suspicious vulnerability functions based on known vulnerability functions included in the function vulnerability library through a pre-trained target function semantic extraction model to determine whether there is a known vulnerability function with the same source as the suspicious vulnerability function in the function vulnerability library, see the following steps 1 to 4:

[0096] Step 1: extract a first function control flow chart corresponding to a known vulnerability function and a second function control flow chart corresponding to a suspicious vulnerability function contained in a function vulnerability library respectively.

[0097] Step 2: Convert the binary assembly code at each node in the first function control flow chart into an intermediate language, and convert the binary assembly code at each node in the second function control flow chart into an intermediate language. In one example, a related tool can be used to convert the binary assembly code into an intermediate language.

[0098] Step 3: Use the pre-trained target function semantic extraction model to perform function similarity matching on the converted first function control flow chart and the second function control flow chart to obtain the function similarity between the known vulnerability function and the suspected vulnerability function. Figure 5 , step three specifically includes:

[0099] (1) Generate a first semantic control flow graph based on the converted first functional control flow graph through an intermediate language unit in a branch, and generate a first embedding vector based on the first semantic control flow graph through a graph embedding unit in the branch.

[0100] (2) generating a second semantic control flow graph based on the converted second functional control flow graph through an intermediate language unit in another branch, and generating a second embedding vector based on the second semantic control flow graph through a graph embedding unit in the branch.

[0101] (3) The distance between the first embedding vector and the second embedding vector is used as the function similarity between the known vulnerability function and the suspected vulnerability function. Specifically, the cosine distance between the first embedding vector and the second embedding vector can be used as the function similarity between the known vulnerability function and the suspected vulnerability function.

[0102] Step 4: If the function similarity between the known vulnerability function and the suspected vulnerability function is greater than a preset similarity threshold, the known vulnerability function is determined to be a known vulnerability function of the same origin as the suspected vulnerability function.

[0103] For specific implementation, see Figure 8 The overall flow chart of a firmware vulnerability detection method based on function semantics is shown. In an embodiment of the present invention, the suspected vulnerability function and the known vulnerability function obtained after function pre-screening are used to extract the function control flow chart. Then, the two function control flow charts are input into the FSEM model to obtain two embedding vectors, and the similarity scores of the two embedding vectors are calculated using cosine similarity. If the similarity scores of the two functions are greater than a preset similarity threshold, the suspected vulnerability function is considered to be a homologous function of the known vulnerability function, and further, it is indicated that the firmware contains the corresponding vulnerability. The embodiment of the present invention combines the binary function control flow chart with the intermediate language to improve the accuracy of binary function similarity detection.

[0104] Based on the above embodiments, the present invention provides a firmware vulnerability detection device based on function semantics, see Fig. 9 The schematic diagram of the structure of a firmware vulnerability detection device based on function semantics is shown, and the device mainly includes the following parts:

[0105] The firmware parsing module 902 is used to perform firmware parsing processing on the IoT firmware to be detected, so as to extract the executable file and its corresponding file information from the IoT firmware to be detected;

[0106] The vulnerability pre-screening module 904 is used to pre-screen the executable file for vulnerability functions based on the file information to obtain suspicious vulnerability functions contained in the IoT firmware to be detected;

[0107] A similarity matching module 906 is used to perform function similarity matching on the suspected vulnerability function based on the known vulnerability functions included in the function vulnerability library through a pre-trained function semantic extraction model, so as to determine whether there is a known vulnerability function with the same source as the suspected vulnerability function in the function vulnerability library;

[0108] The vulnerability detection module 908 is used to determine that the IoT firmware to be detected has a vulnerability when the judgment result of the similarity matching module is yes.

[0109] In one implementation, the similarity matching module 906 is specifically used to:

[0110] Respectively extracting a first function control flow chart corresponding to a known vulnerability function and a second function control flow chart corresponding to a suspected vulnerability function contained in the function vulnerability library;

[0111] Converting the binary assembly code at each node in the first function control flow graph into an intermediate language, and converting the binary assembly code at each node in the second function control flow graph into an intermediate language;

[0112] Through the pre-trained target function semantic extraction model, function similarity matching is performed on the converted first function control flow graph and the second function control flow graph to obtain the function similarity between the known vulnerability function and the suspected vulnerability function.

[0113] In one embodiment, the objective function semantic extraction model includes two branches, each of which includes an intermediate language unit and a graph embedding unit; the similarity matching module 906 is specifically used to:

[0114] Generate a first semantic control flow graph based on the converted first functional control flow graph through an intermediate language unit in a branch, and generate a first embedding vector based on the first semantic control flow graph through a graph embedding unit in the branch;

[0115] and, generating a second semantic control flow graph based on the converted second functional control flow graph through an intermediate language unit in another branch, and generating a second embedding vector based on the second semantic control flow graph through a graph embedding unit in the branch;

[0116] The distance between the first embedding vector and the second embedding vector is used as the function similarity between the known vulnerability function and the suspected vulnerability function.

[0117] In one embodiment, a model training module is further included, which is used to:

[0118] Obtaining a training data set, the training data set includes binary assembly code for training and its corresponding function control flow chart and intermediate language;

[0119] Combine binary assembly codes with the same name under different architectures in the training data set into positive sample function pairs; and combine binary assembly codes with different names in the training data set into negative sample function pairs;

[0120] The positive sample function pairs and the negative sample function pairs are taken as the function pair data set;

[0121] The initial function semantic extraction model is trained, verified and tested using the function pair dataset to obtain the target function semantic extraction model.

[0122] In one implementation, the firmware parsing module 902 is specifically used to:

[0123] Use reverse engineering tools to decompress the IoT firmware to obtain system files;

[0124] Use the specified command to extract the executable file from the system file and extract the file information corresponding to the executable file.

[0125] In one implementation, the similarity matching module 906 is specifically used to:

[0126] Retrieve the file record from the function vulnerability library based on the file information, and determine whether the hash value in the file record is consistent with the hash value corresponding to the executable file;

[0127] If the hash values ​​are consistent, it is determined that the executable file contains the suspicious vulnerability function in the file record;

[0128] If the hash values ​​are inconsistent, the executable file is disassembled to obtain a function set corresponding to the executable file, and it is determined whether the function set corresponding to the executable file contains the suspicious vulnerability function in the file record;

[0129] If the function set corresponding to the executable file does not contain the suspected vulnerability function in the file record, the complement of the function set corresponding to the executable file and the function set in the file record is used as the suspected vulnerability function.

[0130] In one implementation, the similarity matching module 906 is specifically used to:

[0131] If the function similarity between the known vulnerability function and the suspected vulnerability function is greater than a preset similarity threshold, the known vulnerability function is determined to be a known vulnerability function of the same origin as the suspected vulnerability function.

[0132] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0133] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.

[0134] Fig.10 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 10, a memory 11, a bus 12 and a communication interface 13, wherein the processor 10, the communication interface 13 and the memory 11 are connected via the bus 12; the processor 10 is used to execute an executable module stored in the memory 11, such as a computer program.

[0135] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 13 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0136] The bus 12 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0137] Among them, the memory 11 is used to store programs, and the processor 10 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 10 or implemented by the processor 10.

[0138] The processor 10 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 10. The above processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 11, and the processor 10 reads the information in the memory 11 and completes the steps of the above method in combination with its hardware.

[0139] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0140] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0141] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A firmware vulnerability detection method based on function semantics, characterized in that: include: Performing firmware parsing processing on the IoT firmware to be detected, so as to extract an executable file and its corresponding file information from the IoT firmware to be detected; Based on the file information, the executable file is pre-screened for vulnerability functions to obtain suspicious vulnerability functions contained in the IoT firmware to be detected; By using the pre-trained target function semantic extraction model, based on the known vulnerability functions contained in the function vulnerability library, function similarity matching is performed on the suspected vulnerability function to determine whether there is a known vulnerability function with the same source as the suspected vulnerability function in the function vulnerability library; If yes, it is determined that the IoT firmware to be detected has a vulnerability; Through the pre-trained target function semantic extraction model, based on the known vulnerability functions included in the function vulnerability library, the function similarity matching of the suspected vulnerability function is performed, including: respectively extracting a first function control flow chart corresponding to the known vulnerability function included in the function vulnerability library and a second function control flow chart corresponding to the suspected vulnerability function; converting the binary assembly code at each node in the first function control flow chart into an intermediate language, and converting the binary assembly code at each node in the second function control flow chart into an intermediate language; through the pre-trained target function semantic extraction model, the function similarity matching of the converted first function control flow chart and the second function control flow chart is performed to obtain the function similarity between the known vulnerability function and the suspected vulnerability function; The target function semantic extraction model includes two branches, each of which includes an intermediate language unit and a graph embedding unit; through the pre-trained target function semantic extraction model, the converted first function control flow graph and the second function control flow graph are matched for function similarity to obtain the function similarity between the known vulnerability function and the suspected vulnerability function, including: generating a first semantic control flow graph based on the converted first function control flow graph through the intermediate language unit in one of the branches, and generating a first embedding vector based on the first semantic control flow graph through the graph embedding unit in the branch; and generating a second semantic control flow graph based on the converted second function control flow graph through the intermediate language unit in another of the branches, and generating a second embedding vector based on the second semantic control flow graph through the graph embedding unit in the branch; and taking the distance between the first embedding vector and the second embedding vector as the function similarity between the known vulnerability function and the suspected vulnerability function.

2. The firmware vulnerability detection method based on function semantics according to claim 1, characterized in that: The training steps of the objective function semantic extraction model include: Acquire a training data set, wherein the training data set includes binary assembly code for training and its corresponding function control flow chart and intermediate language; Combining binary assembly codes with the same name under different architectures in the training data set into positive sample function pairs; and combining binary assembly codes with different names in the training data set into negative sample function pairs; Taking the positive sample function pairs and the negative sample function pairs as function pair data sets; The function pair data set is used to train, verify and test the initial function semantic extraction model to obtain the target function semantic extraction model.

3. The firmware vulnerability detection method based on function semantics according to claim 1, characterized in that: Performing firmware parsing processing on the IoT firmware to be detected to extract the executable file and its corresponding file information from the IoT firmware to be detected, including: Use reverse engineering tools to decompress the IoT firmware to obtain system files; The executable file is extracted from the system file using a specified command, and the file information corresponding to the executable file is extracted.

4. The firmware vulnerability detection method based on function semantics according to claim 1, characterized in that: Based on the file information, the executable file is pre-screened for vulnerability functions to obtain suspicious vulnerability functions contained in the IoT firmware to be detected, including: Retrieving a file record from a function vulnerability library based on the file information, and determining whether a hash value in the file record is consistent with a hash value corresponding to the executable file; If the Hash values ​​are consistent, it is determined that the executable file contains the suspicious vulnerability function in the file record; If the Hash values ​​are inconsistent, the executable file is disassembled to obtain a function set corresponding to the executable file, and it is determined whether the function set corresponding to the executable file includes the suspicious vulnerability function in the file record; If the function set corresponding to the executable file does not include the suspected vulnerability function in the file record, the complement of the function set corresponding to the executable file and the function set in the file record is used as the suspected vulnerability function.

5. The firmware vulnerability detection method based on function semantics according to claim 1, characterized in that: Determining whether there is a known vulnerability function with the same source as the suspected vulnerability function in the function vulnerability library includes: If the function similarity between the known vulnerability function and the suspected vulnerability function is greater than a preset similarity threshold, the known vulnerability function is determined to be a known vulnerability function of the same origin as the suspected vulnerability function.

6. A firmware vulnerability detection device based on function semantics, characterized in that: include: A firmware parsing module, used for performing firmware parsing processing on the IoT firmware to be detected, so as to extract an executable file and its corresponding file information from the IoT firmware to be detected; A vulnerability pre-screening module, used to perform vulnerability function pre-screening processing on the executable file based on the file information, and obtain suspicious vulnerability functions contained in the IoT firmware to be detected; A similarity matching module is used to perform function similarity matching on the suspected vulnerability function based on known vulnerability functions contained in a function vulnerability library through a pre-trained function semantic extraction model, so as to determine whether there is a known vulnerability function with the same source as the suspected vulnerability function in the function vulnerability library; A vulnerability detection module, used to determine that the IoT firmware to be detected has a vulnerability when the judgment result of the similarity matching module is yes; The similarity matching module is specifically used to: respectively extract a first function control flow chart corresponding to a known vulnerability function contained in a function vulnerability library and a second function control flow chart corresponding to the suspected vulnerability function; convert the binary assembly code at each node in the first function control flow chart into an intermediate language, and convert the binary assembly code at each node in the second function control flow chart into an intermediate language; perform function similarity matching on the converted first function control flow chart and the second function control flow chart through a pre-trained target function semantic extraction model to obtain function similarity between the known vulnerability function and the suspected vulnerability function; The target function semantic extraction model includes two branches, each of which includes an intermediate language unit and a graph embedding unit; the similarity matching module is specifically used to: generate a first semantic control flow graph based on the converted first function control flow graph through the intermediate language unit in one of the branches, and generate a first embedding vector based on the first semantic control flow graph through the graph embedding unit in the branch; and generate a second semantic control flow graph based on the converted second function control flow graph through the intermediate language unit in another of the branches, and generate a second embedding vector based on the second semantic control flow graph through the graph embedding unit in the branch; and use the distance between the first embedding vector and the second embedding vector as the function similarity between the known vulnerability function and the suspected vulnerability function.

7. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 5.

Images