Android malicious software detection method based on API semantic enhancement

Abstract

The invention belongs to the technical field of information security, and relates to an Android malicious software detection method based on API semantic enhancement, and the method comprises the steps: obtaining a to-be-detected Android application; performing static analysis on the to-be-detected Android application, and constructing a function call graph of the to-be-detected Android application; optimizing the function call graph of the Android application; extracting a complete API context sequence of a corresponding function from the optimized function call graph by adopting a recursive algorithm; adopting Skpi-Gram to map the complete API context sequence into feature vectors, and adopting a clustering algorithm to divide the feature vectors with close distances into the same class; inputting the clustered feature vectors into a trained graph convolutional neural network model to obtain a detection result of the to-be-detected Android application; according to the method, the extracted function call graph nodes are optimized by adopting the graph convolutional neural network, so that information loss caused by feature structure conversion is avoided, and the efficiency of model training and the accuracy of a detection result are improved.

Description

Technical Field

[0001] The present invention belongs to the field of information security technology, and in particular relates to an Android malware detection method based on API semantic enhancement. Background Art

[0002] With the continuous development of the communication industry and the increasing popularity of communication networks, mobile smart devices have rapidly entered people's daily lives and are constantly changing people's lifestyles. In recent years, phishing and malware have also become an important issue in the field of Internet security. Attackers are people or organizations with malicious intentions. Their purpose is to access, destroy or steal data in computer systems without authorization. Faced with such a severe Android cyberspace security situation, governments and major Internet companies have taken measures to prevent user information leakage. As a communication and Internet access tool, mobile smart devices contain a lot of private information. At the same time, its popularity has also brought about the rise of mobile payments, which brings convenience while also bringing huge security risks. The software that accounts for the largest proportion of fee consumption and malicious deductions seriously threatens the property safety of users. How to prevent users from being attacked by malware and protect the personal information and property safety of users has important research significance and practical value. Therefore, malware detection has become a hot research direction in the field of Internet security.

[0003] Malware detection is one of the most important topics in the security field. It can be roughly divided into two types according to its development technology route: the first is based on signature code, and the second is based on machine learning. The basic principle of the malware detection method based on signature code is to use the unique feature information of each software for matching. That is, when the malware's specified signature code is known, the malware is detected by matching it with the signature code of the software to be detected. If the same signature code is matched in the existing malware signature code database, the target software is judged as malware, otherwise it is benign software. With the improvement of computer computing power and the development of related machine learning technologies, malware detection methods based on machine learning have begun to become mainstream. The basic principle of malware detection methods based on machine learning is to extract different features to describe the different behaviors of the samples to be analyzed through program analysis and other technologies, and then each sample is represented by a fixed-dimensional vector. Finally, with the help of existing machine learning algorithms, samples with known labels are trained and classifiers are constructed, so that unknown samples can be predicted and judged. Word embedding is used to improve the functional attributes of nodes. Then the main problem of this type of detection technology is that with the continuous evolution of malware variants, the effectiveness of the model will be significantly reduced. In order to deal with the problem of model aging, most existing studies choose to re-label new samples to retrain the aging model. However, this method usually requires a lot of labeling and training costs. Therefore, starting from the API semantics, that is, the attributes of each node should be represented by its adjacent nodes, and word embedding is used to supplement the problem of insufficient feature semantic information in previous solutions, thereby solving the problem of model aging. Summary of the invention

[0004] In order to solve the problems existing in the above prior art, the present invention proposes an Android malware detection method based on API semantic enhancement, which includes: obtaining an Android application to be detected; performing static analysis on the Android application to be detected and constructing a function call graph of the Android application to be detected; optimizing the function call graph of the Android application; using a recursive algorithm to extract the complete API context sequence of the corresponding function from the optimized function call graph; using the trained Skpi-Gram to map the complete API context sequence into an API feature vector, and using the k-means clustering algorithm to divide the API feature vectors with similar distances into the same class family; inputting the clustered API feature vectors into a trained graph convolutional neural network model to obtain the detection result of the Android application to be detected.

[0005] Beneficial effects of the present invention:

[0006] This invention optimizes the extracted function call graph nodes by using a graph convolutional neural network, avoids the information loss caused by feature structure conversion, converts the API into feature vectors, and clusters the differences between API vectors as semantic distances, thereby improving the efficiency of model training and the accuracy of detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is the overall flow chart of the present invention;

[0008] Figure 2 An API embedding diagram in an embodiment of the present invention;

[0009] Figure 3 is an API clustering diagram in an embodiment of the present invention;

[0010] Figure 4 This is a flowchart of the working of the graph convolutional neural network model in the present invention. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] An Android malware detection method based on API semantic enhancement, such as Figure 1 As shown, the method includes: obtaining an Android application to be detected; obtaining an Android application to be detected; performing static analysis on the Android application to be detected and constructing a function call graph of the Android application to be detected; for the function call graph of the Android application, using a function call graph optimization algorithm to retain key node information in the function call graph; for the optimized function call graph, using a recursive algorithm to extract a complete API context sequence of the corresponding function; for the API call sequence, using Skpi-Gram mapping to a feature vector, and using a k-means clustering algorithm to divide API vectors with similar distances into the same class family; inputting the clustered API feature vector into a trained graph convolutional neural network model to obtain the detection result of the Android application to be detected.

[0013] The embodiment of the present invention provides an Android malware detection method based on API semantic enhancement, such as Figure 1 As shown, the following steps are included:

[0014] S1: Get the Android application to be detected;

[0015] S2: Performing static analysis on the Android application to be detected, and constructing a function call graph of the Android application to be detected, wherein the function call graph includes API function nodes and their API call relationships;

[0016] S3: Simplify the function call graph by function graph optimization, classify according to API function categories, set the extracted API function categories, retain only the core API function nodes related to file access and network application, merge other non-core API function nodes, and obtain the function call of the Android application to be detected.

[0017] S4: In order to extract sufficient API semantic information, a recursive algorithm is used to extract the complete API context sequence of the corresponding function. The Skip-Gram model in Word2vec is used for training, and the trained API vectors are clustered.

[0018] S5: Input the adjacency matrix and the feature matrix into the trained graph convolutional neural network model to obtain the detection result of the Android application to be detected.

[0019] In this embodiment, the static analysis process specifically includes:

[0020] S201: Decompile the APK file of the Android application to be detected to generate a code file, and a commonly used decompilation tool is ApkTool;

[0021] S202: extracting the code file to obtain an API call sequence, for example, using AndroidGuard to analyze and extract the decompiled code file (.dex file) to obtain an API call sequence;

[0022] S203: Convert the API call sequence into a call graph structure, perform visualization processing, and obtain a function call graph of the Android application to be detected. A commonly used conversion tool is the Gephi tool.

[0023] Android system is often translated as Android system, and Android application or Android app is often translated as Android application.

[0024] APK file (full name: Android application package) is an application package file format used by the Android operating system to distribute and install mobile applications and middleware. If an Android application code wants to run on an Android device, it must first be compiled and then packaged into a file that can be recognized by the Android system before it can be run. The file format that can be recognized and run by the Android system is "APK". An APK file contains compiled code files (.dex files), file resources (resources), native resource files (assets), certificates (certificates), and manifest files (manifestfile).

[0025] When an Android application is packaged, aapt is used to compile the resource files in the project. Generally, after opening the APK file, all the resource files are compiled. These files are basically unreadable and incomprehensible.

[0026] ApkTool is often used to decompile resource files in Android. The ApkTool tool can decompile resource files compiled in Android projects.

[0027] Androguard is a full-featured Python toolset for analyzing and reverse engineering Android files. Androguard provides a variety of commands and options for working with different types of Android files and tasks.

[0028] In this embodiment, the process of optimizing the function call graph includes:

[0029] S301: Obtain the function call graph, traverse all API function nodes in the function call graph in sequence, and determine whether the API function node is a sensitive API function node in sequence according to the data set in the sensitive API, otherwise jump to step three.

[0030] S302: If the API function node is a sensitive API function node, further determine the function category of the API function node based on the class library package provided by the Android platform. If the function category is associated with file management and network access, mark it and jump to step S304.

[0031] S303: If the API function node is a non-sensitive API function node or a non-sensitive category node, merge it, traverse the adjacent nodes of the API function node in sequence, and determine whether the adjacent node is a marked node. If it is a marked node, skip traversing the next adjacent node. Otherwise, continue to determine whether the adjacent node is a sensitive category node. If it is a sensitive category node, mark it, otherwise merge the adjacent node with the API function node;

[0032] S304: Repeat steps S301-S303 until the current API function node is the last API function node in the function call graph, then stop calculating to obtain the function abstract graph of the Android application to be detected.

[0033] Word2vec is a lexical semantic analysis model proposed by Google researchers. It uses corpus data as input and output to train a neural network model, converting words into digital vectors, thereby mapping high-dimensional word vectors to specified low-dimensional i-dimensional word embedding vectors. The model it uses contains three different layers, namely the input layer, projection layer, and output layer, and models the corpus with a three-layer structure. Currently, Word2vec has two different model training methods, namely the CBOW model and the Skip-gram model. The CBOW model takes the context sequence of the currently selected word as input and calculates the probability distribution of the current word.

[0034] The algorithm first traverses all API function nodes in the function call graph. For each node, the algorithm determines whether it is a known sensitive API, which constitutes a large set of 21,986 members. By comparing with this set, the algorithm can identify sensitive API nodes. Once a node is confirmed as a sensitive API, the algorithm further analyzes its functional category, mainly based on the class library package of the Android platform. If the API function is related to file operations or network communication, the node will be retained and marked. For those sensitive API nodes that do not involve files and networks, the algorithm will perform a merge operation. Specifically, the algorithm will check the adjacent nodes of these nodes, and if they have been marked, the algorithm will ignore them. If the adjacent nodes are non-sensitive, the algorithm will merge them with the original node. Through these steps, the algorithm generates a simplified function abstract graph, which not only simplifies the structure, but also improves the concentration of sensitive information, thereby improving the efficiency and accuracy of model training. The advantage of this method is that it can identify sensitive APIs and can perform targeted processing based on the functions of the API, which helps to reduce unnecessary information while retaining data that is critical to analyzing malicious application behavior. This plays a significant role in distinguishing malicious and benign applications and improving the accuracy of security analysis.

[0035] The number of nodes in the function call graph is simplified, non-core nodes are merged, and some nodes with relatively low safety factors are merged to obtain a function abstract graph. In this way, the complexity of the graph can be reduced and the training efficiency of the later model can be improved.

[0036] In this embodiment, if Figure 2 and Figure 3 As shown in the figure, the specific steps of API embedding and API clustering include:

[0037] S501: First, embed each API into [0,1] according to the position of the API in the context of a set of applications. k In the n-dimensional vector.

[0038] S502: Build an API vocabulary consisting of all unique APIs in the path.

[0039] S503: Export all neighboring APIs that are at most S APIs away from the target API in the API path, pair each target API with each of its neighboring APIs, and use these API pairings to train the Skip-Gram model.

[0040] S504: adopting a k-means clustering algorithm, which divides the API vector into k clusters, each API vector belongs to the cluster with the closest average value, and the maximum size difference of all clusters does not exceed one.

[0041] Word2vec is a lexical semantic analysis model proposed by Google researchers. It uses corpus data as input and output to train a neural network model, converting words into digital vectors, thereby mapping high-dimensional word vectors to specified low-dimensional i-dimensional word embedding vectors. The model it uses contains three different layers, namely the input layer, projection layer, and output layer, and models the corpus with a three-layer structure. Currently, Word2vec has two different model training methods, namely the CBOW model and the Skip-gram model. The CBOW model takes the context sequence of the currently selected word as input and calculates the probability distribution of the current word.

[0042] This paper uses the Skip-Gram model in Word2vec for training. Skip-Gram can capture the semantic relationship between words well, performs well in word vector learning, and has significant advantages in processing low-frequency word sparse data and polysemous words. The Skip-Gram model consists of an input layer, a hidden layer, and an output layer. In this model, a real-number input hidden matrix W v*k Used to convert the input vector into a hidden-input vector, a real-valued hidden-output matrix W` v*kUsed to further transform the hidden input vector into an output vector, where V is the API vocabulary size and K is the size of the API vector. During training, the target API and all its paired APIs are considered as the input and target output of the Skip-Gram model, respectively. For each target API, the Skip-Gram model calculates the input-hidden matrix, the hidden-output matrix, and the softmax function, and then uses backpropagation to update the elements in the two matrices to minimize the error between the calculated output and the target output.

[0043] In this implementation example, the specific training steps of the graph convolutional neural network model include:

[0044] S601: Acquire training samples, where the training samples are divided into a training set and a validation set;

[0045] S602: Input the training set and the validation set into the GCN model. The parameters input into the input layer of the GCN model are the batch, node and feature of the training set and the validation set, where batch is the number of graph structures in a single training batch, node is the number of nodes in the graph structure, and feature is the dimension of the initial feature of the node. In the embodiment of the present invention, after the processing of steps S2-S4, feature is N, node is K, and batch is 1.

[0046] S603: constructing a label pair according to the training set and the validation set, taking the input training set as an anchor sample, randomly obtaining samples with the same label from the training set and the validation set as positive samples, and samples with different labels as negative samples, and using these three samples to construct a label pair;

[0047] S604: Calculate the loss function of the GCN model, where the loss function is a triplet loss. Back propagate the loss function to optimize the model. When the loss function converges or reaches a preset condition, stop training to obtain a trained GCN model.

[0048] Traditional machine learning models such as convolutional neural networks (CNN) are non-Euclidean structures and cannot directly process function call graphs. Therefore, the embodiment of the present invention adopts a graph convolutional neural network (GCN) as a model to avoid information loss caused by feature structure conversion and affect the detection results. The graph convolutional neural network GCN consists of three layers: the first layer is the input layer; the second layer is the hidden layer, which is composed of multiple convolution layers; the third layer is the output layer, which is composed of a pooling layer and a fully connected multi-layer perceptron classifier that can generate global information. In the hidden layer, the main function of the convolution layer is to extract the main features of the data and reduce the size of the data to achieve the purpose of dimensionality reduction. The pooling layer can realize maximum pooling or average pooling according to the set rules, thereby reducing the dimension of the data; it can also enable the network model to obtain the ability to resist small displacement and deformation. The convolution and pooling method greatly reduces the complexity of the model, reduces the various parameters of the model, retains the key features of the original data, and avoids the accuracy loss caused by model conversion due to the inability to directly process the graph structure.

[0049] In this embodiment, if Figure 4 As shown, the model testing process of the GCN neural network model includes:

[0050] S701: Obtain a test set, and input the test set into a GCN neural network model;

[0051] S702: Send the data obtained by the hidden layer to the pooling layer to obtain the output data of the pooling layer. The fully connected layer maps the output data of the pooling layer to classification numbers, where the classification numbers are 0 and 1 respectively, label 0 is a benign sample, and label 1 is a malignant sample, to obtain the classification result of the test set;

[0052] S703: According to the classification result of the test set (i.e., the performance of the evaluation model), if the expected test goal is achieved (i.e., the model is evaluated as having good performance at this time), the model training is stopped to obtain the trained graph convolutional neural network model; if the expected test goal is not achieved (i.e., the model is evaluated as having insufficient performance or not up to standard at this time), the GCN model parameters and the number of training samples are adjusted, and the GCN model is retrained until the expected test goal is achieved, then the TextCNN model training is stopped to obtain the trained GCN neural network model.

[0053] The training samples in the embodiments of the present invention mainly use Android benign applications obtained from third-party platforms such as Google Play and App Store. The obtained applications need to be input into VirusTotal and are included only after being analyzed and scanned to be normal; the malicious applications used are from Android malicious sample sets, which are obtained from Drebin sample sets and AMD sample sets. VirusTotal is a website that provides free suspicious file analysis services. The Drebin sample set contains 5,560 samples from 179 Android malware families collected from August 2010 to October 2013. The AMD sample set is widely used because of its large number of samples and clear family classification. AMD contains a total of 24,553 samples collected from 2012 to 2018, including 71 Android malware families and 135 variants. This data set basically covers the latest situation of current Android malware and continues to be open to the public. The package includes many different types of applications, including social communication, office software, game entertainment, shopping, video, reading, and life software. The number of applications in each category is kept at around 15% to make the features learned by the algorithm diverse and accurately measure the performance of the model in the real application environment.

[0054] A total of 453 malicious Android applications were extracted from the Drebin sample set and the AMD sample set, and 1421 benign Android applications were obtained from third-party platforms, totaling 1874 Android applications. These Android applications were processed through the steps S2-S4 to obtain the function abstract graph of each Android application and its corresponding feature matrix, which constituted the training samples of the embodiment of the present invention, of which 90% were used as training sets and validation sets, and the remaining 10% were used as test sets. The trained GCN model was tested using the test set. If the expected test target was not achieved (i.e., the model was evaluated as not performing well or not meeting the standard), the GCN model parameters and the number of training samples were adjusted, and the GCN model was retrained.

[0055] Use the trained GCN model classifier for predictive detection, input the test set into the model, and finally complete the classification of the final result, that is, the generation of labels, through the pooling layer. The final result label 0 is a benign sample, and label 1 is a malignant sample, completing the model's detection work.

[0056] The API sequence of this application can be directly input into the deep neural network for training, but the deep neural network regards these APIs as meaningless character inputs and does not know the semantics. As a result, the API semantic information cannot be learned during the model training process. In order to solve this problem, a method based on API semantic embedding is proposed, which combines the native API and API word segmentation representation vectors. Through graph neural networks, word embedding is used to improve the functional attributes of nodes, that is, the attributes of each node should be represented by its adjacent nodes. Word embedding is used to supplement the problem of insufficient feature semantic information in previous solutions. At the same time, a graph convolutional neural network is used to further process the extracted feature graph, avoiding the information loss caused by the feature structure conversion, effectively improving the efficiency of model training and the accuracy of detection results, which is of great significance to the current field of Android malware detection for large data sets.

[0057] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, disk or CD, etc.

[0058] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An Android malware detection method based on API semantic enhancement, characterized in that: include: Get the Android application to be tested; Perform static analysis on the Android application to be tested and construct a function call graph of the Android application to be tested; Optimize the function call graph of Android applications; A recursive algorithm is used to extract the complete API context sequence of the corresponding function from the optimized function call graph; the trained Skpi-Gram is used to map the complete API context sequence into an API feature vector, and the k-means clustering algorithm is used to divide the API feature vectors with similar distances into the same category; the clustered API feature vectors are input into the trained graph convolutional neural network model to obtain the detection result of the Android application to be detected.

2. The method for detecting Android malware based on API semantic enhancement according to claim 1, characterized in that: The static analysis of the Android application to be detected includes: decompiling the APK file of the Android application to be detected to obtain a code file; and analyzing the function call instructions in the code file to construct a function call graph.

3. The method for detecting Android malware based on API semantic enhancement according to claim 1, characterized in that: Optimizing the function call graph of the Android application includes: simplifying the function call graph by means of function graph optimization, classifying according to API function categories, setting the extracted API function categories, retaining core API function nodes related to file access and network application, merging other non-core API function nodes, and obtaining the function call graph of the Android application to be detected.

4. The method for detecting Android malware based on API semantic enhancement according to claim 1, characterized in that: Extracting the complete API context sequence of the corresponding function through a recursive algorithm includes: obtaining the API call sequence; embedding each API into [0,1] according to the context of the API in a set of call sequences; k The complete API context sequence is obtained from the n-dimensional vector.

5. The method for detecting Android malware based on API semantic enhancement according to claim 1, characterized in that: Training the Skip-Gram includes: building an API vocabulary consisting of all unique APIs in the API call sequence; deriving all adjacent APIs that are at most S APIs away from the target API in the API path, and pairing each target API with its adjacent APIs to train the Skip-Gram model.

6. The method for detecting Android malware based on API semantic enhancement according to claim 1, characterized in that: Training the graph convolutional neural network model includes: obtaining training samples, wherein the training samples are divided into a training set and a validation set; inputting the data in the training set into the graph convolutional neural network model for training; and the test set is used to test the trained graph convolutional neural network model.

7. The method for detecting Android malware based on API semantic enhancement according to claim 6, characterized in that: The data in the training set are batch, node and feature, where batch is the number of graph structures input in a single training batch, node is the number of nodes in the graph structure, and feature is the dimension of the initial feature of the node.

8. The method for detecting Android malware based on API semantic enhancement according to claim 6, characterized in that: Testing the graph convolutional neural network model includes: obtaining a test set and inputting the test set into the graph convolutional neural network model; sending the data obtained from the hidden layer to the pooling layer to obtain the output data of the pooling layer, and the fully connected layer maps the output data of the pooling layer to classification numbers, where the classification numbers are 0 and 1 respectively, label 0 is a benign sample, and label 1 is a malignant sample, to obtain the classification result of the test set; according to the classification result, if the expected test goal is achieved, stop training the model to obtain the trained graph convolutional neural network model; if the expected test goal is not achieved, adjust the model parameters and the number of training samples, and retrain the model until the expected test goal is achieved, then stop training the model to obtain the trained graph convolutional neural network model.

Images