Android malware detection and classification method based on multi-feature fusion deep learning

Abstract

This invention discloses a method for detecting and classifying Android malware based on multi-feature fusion deep learning. The method is accomplished by obtaining a sample of an APK to be tested, feeding the sample into a trained model, and obtaining detection results. The invention uses deep learning to capture the data flow patterns of different types of malware from static taint paths. The taint paths are used as a set of features for detecting and classifying Android malware. Feature fusion is performed using a Wide & Deep model, where the Wide portion processes two types of features: sensitive API calls and dangerous permissions, and the Deep portion processes static taint paths. The fused features add semantic information from the static taint paths, enabling the learning of behavioral patterns of different malware, improving the accuracy and robustness of the model.

Description

Technical Field

[0001] The present invention belongs to the field of software engineering and relates to an Android malware detection method, and specifically to an Android malware detection and classification method based on multi-feature fusion deep learning. Background Art

[0002] With the popularity of smart phones, various mobile applications such as navigation systems, chat software, shopping software, etc., allow users to travel, chat, pay and other functions through their mobile phones, greatly facilitating people's lives.

[0003] However, behind this convenience comes the risk of illegal theft of users' photos, travel habits, and personal assets stored on various mobile apps. Attackers are targeting mobile phone users, developing various malware to steal private user data such as chat logs, text messages, address books, and bank accounts, potentially causing significant losses.

[0004] Faced with these challenges, countries around the world have successively introduced regulations to protect citizens' personal information and regulate the collection, transmission, and use of user information by network operators. Major companies such as Huawei, Google, and Tencent are actively taking measures to protect user privacy, developing their own mobile app review tools to conduct large-scale mobile malware detection, aiming to identify and address malware before it reaches the market.

[0005] Static analysis-based methods do not require running the source program and are suitable for large-scale malware detection. Existing static analysis-based methods have the following characteristics:

[0006] 1) Traditional static detection methods do not require running the source program and analyze information such as data flow and control flow within the program to achieve the purpose of detecting malware. However, this method has disadvantages such as low accuracy and high false positive rate.

[0007] 2) Static detection methods based on deep learning do not require running the source program. Instead, they use reverse engineering to obtain static features of Android programs, such as API calls, permissions, and opcodes. One or more of these features are then fed into a deep learning model to train a classifier and perform detection. These methods often use API calls and permissions for malware detection, but these features are often isolated and lack semantic information about the program, making them vulnerable to malware poisoning attacks.

[0008] Existing research on the above two methods is mostly independent, without considering the integration of APIs, permissions and the complete taint propagation path. In addition, APIs and permissions are mostly represented by discrete vectors, lacking program semantic information, and without considering the classification of malware behavior. Summary of the Invention

[0009] To address the aforementioned issues in the background technology, the present invention provides an Android malware detection and classification method based on multi-feature fusion deep learning. This method combines static taint analysis techniques from traditional static detection methods with deep learning techniques. It fuses three features: sensitive API calls, dangerous permissions, and static taint paths. This complements the semantic information of a program's static taint paths and can distinguish different malicious behaviors. Therefore, linking static taint analysis with deep learning through feature fusion is a feasible approach for detecting and classifying Android malware.

[0010] The purpose of the present invention is achieved through the following technical solutions:

[0011] A method for detecting and classifying Android malware based on multi-feature fusion deep learning includes the following steps:

[0012] S1: Construct an Android malware detection dataset and divide the data in the dataset into training set, validation set and test set;

[0013] S2: Identify the shell program for the Apk files in the training set. If the Apk has a shell, unshell it and add the unshelled Apk to T. If the Apk is not unshelled, add it directly to T to obtain the unshelled training set T.

[0014] S3: Perform reverse analysis on the Apk files in the training set T to extract the API calls and permissions of each Apk; compare the extracted API calls and permissions with the sensitive API call and dangerous permission lists to obtain the sensitive API call feature vector α and the dangerous permission feature vector β;

[0015] S4: Perform static taint analysis on the Apk in the training set T and preprocess the results to obtain the static taint path file δ for each Apk, and then train the vocabulary W based on all δ;

[0016] S5: Generate the vector representation η of each Apk static taint path according to the vocabulary W, and use the deep learning model to obtain the static taint path feature vector γ;

[0017] S6: Fusion of three types of features: dangerous permissions, sensitive API calls, and static taint paths. The fused vector is used as the final representation of an Apk and connected to the output layer node. The Wide&Deep model is trained using the training set T. When the training loss no longer decreases, the model training is stopped. The validation set is used to adjust the hyperparameters of the Wide&Deep model to achieve the optimal training effect of the Wide&Deep model. Finally, the test set is used to test the generalization error.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] The present invention fully considers the impact of static taint paths on program behavior classification. Through deep learning, the data flow patterns of different types of malware can be captured from static taint paths. Therefore, the taint paths are used as a set of features to detect and classify Android malware. Relying on feature fusion, the static taint analysis technology in traditional program analysis technology is combined with deep learning. The model uses Wide&Deep, where the Wide part processes two types of features: sensitive API calls and dangerous permissions, and the Deep part processes static taint paths. Compared with using only sensitive APIs and dangerous permissions, the fused features increase the semantic information carried by static taint paths. Compared with using only static taint analysis, the abstract expression ability of deep learning is added, which improves the accuracy and reduces the false alarm rate. The two complement each other, improving the accuracy of Android malware detection while also improving the robustness of the model, and can classify and identify common malicious applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the Android malware detection and classification method based on multi-feature fusion deep learning of the present invention.

[0021] Figure 2 It is a schematic diagram of extracting static stain paths. DETAILED DESCRIPTION

[0022] The technical solution of the present invention is further described below with reference to the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention that does not depart from the spirit and scope of the technical solution of the present invention should be included in the scope of protection of the present invention.

[0023] The present invention provides an Android malware detection and classification method based on multi-feature fusion deep learning, which is completed by obtaining an APK sample to be tested, feeding the APK sample to be tested into a trained model and obtaining a detection result. Figure 1 As shown, the specific steps include:

[0024] S1: Construct an Android malware detection dataset and divide the data in the dataset into training, validation, and test sets. The specific steps are as follows:

[0025] S101: Collect Android malware and non-malware;

[0026] S102: Malware is classified into four categories based on different behaviors: advertising malware, mobile banking malware, short message service malware, and risky software; the dataset used is from Samaneh Mahdavifar et al. (Mahdavifar, Samaneh, et al. Dynamic android malware category classification using semi-supervised deep learning. 2020 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on CyberScience and Technology Congress (DASC / PiCom / CBDCom / CyberSciTech). IEEE, 2020.), (Mahdavifar, Samaneh, Dima Alhadidi, and Ali Ghorbani. Effective and efficient hybrid android malware classification using pseudo-label stacked auto-encoder. Journal of Network and Systems Management 30.1(2022):1-34.) is constructed, and the Android programs are divided according to their behaviors in the data. The dataset can be expanded according to the above principles;

[0027] S103: Divide the above four types of malware and non-malware into a training set, a validation set, and a test set according to a ratio of 6:2:2.

[0028] S2: Identify the shell program for the Apk files in the training set. If the Apk has a shell, unshell it and add the unshelled Apk to T. If the Apk is not unshelled, add it directly to T to obtain the unshelled training set T. The specific steps are as follows:

[0029] S201: Construct an empty set T. For each Apk sample in the training set, use the GDA tool to identify the packed program. If it is not packed, add this Apk directly to T. If it is packed, proceed to the next step.

[0030] S202: Obtain the unpacking point of the Apk through the IDA tool;

[0031] S203: Write corresponding hook code according to the unpacking point to unpack the program, and add the unpacked Apk to T.

[0032] S3: Perform reverse analysis on the Apk files in the training set T to extract the API calls and permissions of each Apk; compare the extracted API calls and permissions with the sensitive API call and dangerous permission lists to obtain the sensitive API call feature vector α and the dangerous permission feature vector β. The specific steps are as follows:

[0033] S301: Construct a sensitive API call list and a dangerous permission list based on sensitive API calls and dangerous permissions respectively. The sensitive API call and dangerous permission lists use the 61 dangerous permissions and 97 sensitive API calls used in adversarial samples by Chen S et al. (Chen S, Xue M, Fan L, et al. Automated poisoning attacks and defenses in malware detection systems: An adversarial machine learning approach[J]. computers & security, 2018, 73: 326-344.);

[0034] S302: Use Android Guard to perform reverse analysis on each Apk in the training set T, extract API calls through the intermediate representation Smali, and extract requested permissions through AndroidManifest.xml;

[0035] S303: For all extracted API calls, check whether they are in the sensitive API call list. If so, set the corresponding position to 1, otherwise it is set to 0, and obtain a 97-dimensional sensitive API call feature vector α; for all extracted permissions, check whether they are in the dangerous permission list. If so, set the corresponding position to 1, otherwise it is set to 0, and obtain a 61-dimensional dangerous permission feature vector β.

[0036] S4: Perform static taint analysis on the Apks in the training set T to obtain the static taint path file δ for each Apk, and use all static taint path files to train the vocabulary W. The specific steps are as follows:

[0037] S401: Construct source and sink lists based on Susi. Susi is a rule-based method for constructing Android sources and sinks proposed by Siegfried Rasthofer et al. (Rasthofer, Siegfried, Steven Arzt, and Eric Bodden. A machine-learning approach for classifying and categorizing android sources and sinks. NDSS. Vol. 14. 2014).

[0038] S402: For each Apk in the training set T, use FlowDroid to perform static taint analysis to obtain the static taint path of each Apk. FlowDroid is a static taint analysis tool for Android programs proposed by Steven Arzt et al. (Arzt, Steven, et al. Flowdroid: Precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps. Acm Sigplan Notices 49.6 (2014): 259-269.). It can obtain the static taint path of the program based on predefined sources and sinks.

[0039] S402: Perform pre-processing operations such as word segmentation, symbol removal, and removal of low-frequency words with a number of appearances less than 5 on the extracted static taint path to obtain a static taint path file δ. The static taint path extraction process is as follows: Figure 2 As shown;

[0040] S403: For the preprocessed static taint path file δ, use the Word2Vec algorithm to train the vocabulary and use the CBOW bag-of-words model to obtain the vocabulary W. Word2Vec is a language model proposed by Tomas Mikolov et al. (Mikolov, Tomas, et al. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).), where CBOW predicts the current word based on the context. Here, the word vector dimension for training is set to 100, and the number of iterations is set to 5.

[0041] S5: Generate the vector representation η of each Apk static taint path based on the vocabulary W and use the BiLSTM model to obtain the static taint path feature vector γ. The specific steps are as follows:

[0042] S501: Arrange the static taint path file δ of each Apk in order, obtain the embedding representation η according to the trained vocabulary W, and control the length of the static taint path file δ of each Apk to be 1000. The part exceeding 1000 is discarded, and the part less than 1000 is padded with zeros;

[0043] S502: The vector representation η of the static taint path is used as the input of the BiLSTM layer. The BiLSTM layer includes 100 BiLSTM nodes. After the output of the BiLSTM layer, a 200-dimensional static taint path feature vector γ is obtained.

[0044] S6: Fuse the three types of features: dangerous permissions, sensitive API calls, and static taint paths. The fused vector is used as the final representation of an Apk. The dimension of the fused feature is 358 and is connected to the output layer node. Use the training set T to train the Wide & Deep model. When the training loss no longer decreases, stop the model training and use the validation set to adjust the hyperparameters of the Wide & Deep model to achieve the optimal training effect of the Wide & Deep model. Finally, use the test set to test the generalization error. The specific steps are as follows:

[0045] S601: Take the sensitive API call feature vector α, dangerous permission feature vector β, and static taint path file δ of the Apk as input to the Wide & Deep model;

[0046] S602: Use the Wide & Deep model to fuse features. The input of the Wide part is the sensitive API call feature vector α and the dangerous permission feature vector β of the Apk, and the input of the Deep part is the static taint path file δ after preprocessing of the Apk. The Wide & Deep model was proposed by Heng-Tze Cheng et al. (Cheng, Heng-Tze, et al. Wide & deep learning for recommender systems. Proceedings of the 1st workshop on deep learning for recommender systems. 2016.) and combines the advantages of linear models and deep neural networks.

[0047] S603: Concatenate the feature vectors α and β output by the wide part and the feature vector γ output by the deep part to obtain fused features, and fully connect the fused features with the five nodes of the output layer to obtain the classification result;

[0048] S604: Train the model on the training set T until the training loss stops decreasing, and use the validation set to adjust the hyperparameters of the Wide&Deep model to optimize the training effect of the Wide&Deep model. Finally, use the test set to test the generalization error. Let L represent the training loss. For n samples, y i represents the true label of the i-th sample, Represents the predicted label of the i-th sample, then the loss function expression used by the Wide&Deep model is:

[0049]

[0050] This invention fully considers the impact of the semantic information of static taint paths on program behavior classification. By combining deep learning with static taint analysis technology through feature fusion, it not only identifies malware but also effectively distinguishes the behaviors of different types of malware. By integrating the semantic information of static taint paths with sensitive APIs and dangerous permissions, this invention not only increases the accuracy and robustness of the model but also enables the classification and identification of common malicious applications.

[0051] Example:

[0052] For a short message service malware x, extracting sensitive API calls results in a vector α = [0, 1, 1, 0, …, 1] with a dimension of 97, and extracting dangerous permissions results in a vector β = [1, 0, 0, 1, …, 1] with a dimension of 61.

[0053] Performing taint analysis on it using FlowDroid will obtain its static taint path, which is an Xml file. Preprocessing it will obtain the static taint path file δ, which is one or more static taint paths consisting of Source to Sink.

[0054] α, β, and δ are used as input to the detection model. The tainted path length of δ is limited to 1000 words, and redundant words are truncated. Feature fusion is performed using Wide & Deep to generate a 358-dimensional feature vector. The classification probabilities obtained from the five classification nodes in the output layer are compared, and the label with the highest probability is selected as the predicted label. Ultimately, x is successfully classified as Short Message Service malware.

Claims

1. A method for detecting and classifying Android malware based on multi-feature fusion deep learning, characterized by The method comprises the following steps: S1: Construct an Android malware detection dataset and divide the data in the dataset into training set, validation set and test set; S2: Identify the shell program for the Apk files in the training set. If the Apk has a shell, unshell it and add the unshelled Apk to T. If the Apk is not unshelled, add it directly to T to obtain the unshelled training set T. S3: Perform reverse analysis on the Apk files in the training set T to extract the API calls and permissions of each Apk; compare the extracted API calls and permissions with the sensitive API call and dangerous permission lists to obtain the sensitive API call feature vector α and the dangerous permission feature vector β; S4: Perform static taint analysis on the Apks in the training set T and preprocess the results to obtain the static taint path file δ for each Apk, and then train the vocabulary W based on all δ; S5: Generate the vector representation η of each Apk static taint path according to the vocabulary W, and use the deep learning model to obtain the static taint path feature vector γ; S6: Fusion of three types of features: dangerous permissions, sensitive API calls, and static taint paths. The fused vector is used as the final representation of an Apk and connected to the output layer node. The Wide & Deep model is trained using the training set T. When the training loss no longer decreases, the model training is stopped. The validation set is used to adjust the hyperparameters of the Wide & Deep model to achieve the optimal training effect of the Wide & Deep model. Finally, the test set is used to test the generalization error. The specific steps are as follows: S601: Take the sensitive API call feature vector α, dangerous permission feature vector β, and static taint path file δ of the Apk as input to the Wide & Deep model; S602: Use the Wide & Deep model to fuse features, where the input of the Wide part is the sensitive API call feature vector α and the dangerous permission feature vector β of the Apk, and the input of the Deep part is the static taint path file δ after preprocessing of the Apk; S603: Concatenate the feature vectors α and β output by the wide part and the feature vector γ output by the deep part to obtain fused features, and fully connect the fused features with the five nodes of the output layer to obtain the classification result; S604: Train the model on the training set T until the training loss stops decreasing, then stop the model training. Use the validation set to adjust the hyperparameters of the Wide & Deep model to optimize the training effect of the Wide & Deep model. Finally, use the test set to test the generalization error.

2. The Android malware detection and classification method based on multi-feature fusion deep learning according to claim 1 is characterized in that The specific steps of S1 are as follows: S101: Collect Android malware and non-malware; S102: Malware is divided into four categories based on different behaviors: advertising malware, mobile banking malware, short message service malware, and risky software; S103: Divide the above four types of malware and non-malware into a training set, a validation set, and a test set.

3. The Android malware detection and classification method based on multi-feature fusion deep learning according to claim 1 is characterized in that The specific steps of S2 are as follows: S201: Construct an empty set T. For each Apk sample in the training set, use the GDA tool to identify the packed program. If it is not packed, add this Apk directly to T. If it is packed, proceed to the next step. S202: Obtain the unpacking point of the Apk through the IDA tool; S203: Write corresponding hook code according to the unpacking point to unpack the program, and add the unpacked Apk to T.

4. The Android malware detection and classification method based on multi-feature fusion deep learning according to claim 1 is characterized in that The specific steps of S3 are as follows: S301: Build a sensitive API call list and a dangerous permission list based on sensitive API calls and dangerous permissions respectively; S302: Use Android Guard to perform reverse analysis on each Apk in the training set T, extract API calls through the intermediate representation Smali, and extract requested permissions through AndroidManifest.xml; S303: For all extracted API calls, check whether they are in the sensitive API call list. If so, set the corresponding position to 1, otherwise set it to 0, and obtain the sensitive API call feature vector α; for all extracted permissions, check whether they are in the dangerous permission list. If so, set the corresponding position to 1, otherwise set it to 0, and obtain the dangerous permission feature vector β.

5. The Android malware detection and classification method based on multi-feature fusion deep learning according to claim 1 is characterized in that The specific steps of S4 are as follows: S401: Build Source and Sink lists based on Susi; S402: For each Apk in the training set T, use FlowDroid to perform static taint analysis, preprocess the analysis results, and obtain the static taint path file δ; S403: For the static taint path file δ, use the Word2Vec algorithm to train the vocabulary and use the CBOW bag-of-words model to obtain the vocabulary W.

6. The Android malware detection and classification method based on multi-feature fusion deep learning according to claim 1 is characterized in that The specific steps of S5 are as follows: S501: Get the vector representation η of the static taint path file δ of each Apk according to the trained vocabulary W, and control the length of the static taint path δ of each Apk to be 1000. The part exceeding 1000 is discarded, and the part less than 1000 is padded with zeros; S502: Use η as the input of the BiLSTM layer. The BiLSTM layer includes 100 BiLSTM nodes. The output after the BiLSTM layer is the static taint path feature vector γ.

Images