A robust detection method for Android malicious applications based on hybrid features

Abstract

This invention relates to a robust detection method for Android malicious applications based on hybrid features, comprising the following steps: 1. Collecting a malware dataset; 2. Extracting and fusing traffic behavior features and dynamic runtime features, and converting the mixed features into a grayscale image; 3. Building a deep learning framework, in which the deep learning model used for training is the EfficientNet v2+Attention model; 4. Repeating multiple iterations to obtain the final prediction result. This invention uses a feature fusion method combining traffic behavior features and dynamic runtime features, and through model training, it can significantly improve the robustness of malware classification.

Description

Technical Field

[0001] This invention relates to the field of methods for detecting Android malicious applications, and specifically to a robust detection method for Android malicious applications based on hybrid features. Background Technology

[0002] In recent years, with the rapid development of mobile internet and the widespread use of smartphones in various security-sensitive operations, a large number of Android applications have emerged. Cyberattacks and cybercrime using malware have increased dramatically. Malware is a computer program that infiltrates and damages a computer without the user's consent. Due to Android's large market share and its open-source development ecosystem, many malware developers upload their malicious applications to third-party marketplaces, and even Google's official marketplace, by evading detection provided by Android sandboxes or other existing antivirus mechanisms. In the Android operating system, applications can be easily decompiled for code analysis. In legitimate contexts, obfuscation and hardening can effectively prevent attackers from copying or even modifying the source code; however, in illegitimate contexts, attackers can use hardening to build highly disguised malware to evade security systems or security software checks.

[0003] The booming development of mobile applications, especially on the Android platform, has been accompanied by a series of problems, one of which is the balance between intellectual property protection and security. To protect intellectual property and prevent malicious theft and reverse engineering of applications, developers have begun to use hardening techniques to enhance application security. Hardening techniques make application code more difficult to decompile and reverse engineer, thus protecting developers' intellectual property and trade secrets. While hardening techniques have achieved significant results in protecting intellectual property, they have also raised new problems. Criminals have begun to abuse hardening techniques, using them in the development of malware. This situation poses a serious security threat to the mobile application ecosystem. Malicious developers use hardening techniques to hide malicious code, making it difficult to detect by traditional malware detection methods, thereby increasing the difficulty of malware propagation and concealment.

[0004] Machine learning-based methods are also widely used in malware detection. These methods train models to identify patterns of malicious behavior and can adapt to constantly changing malware forms. However, existing malware detection methods and models may fail when faced with malware that exhibits adversarial analysis features (such as hardened malware). Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a robust detection method for Android malicious applications based on hybrid features, in response to the above problems and requirements.

[0006] To solve the above technical problems, the present invention adopts the following technical solution:

[0007] A robust detection method for hardened Android malicious applications based on hybrid features includes the following steps:

[0008] Step 1: Establish a dataset containing various benign and malicious Android applications, strengthen the signature of the dataset, and divide it into training set, verification set and test set. The training set and verification set are used for training, and the test set is used for testing.

[0009] Step 2: Obtain dynamic runtime features and traffic data features for the samples in the dataset;

[0010] Step 3: Concatenate the traffic features of each sample with the dynamic runtime features to form a fused call sequence feature, and convert it into a grayscale image;

[0011] Step 4: Construct a deep learning classification model. The deep learning classification model integrates a custom attention mechanism. The output of the deep learning classification model is the classification category of the input data. Input the grayscale image corresponding to the sample in the training set into the deep learning classification model to train the deep learning classification model. Then, use the validation set to validate the model to obtain the trained classification model.

[0012] Step 5: Test the trained classification model obtained in the previous step using the test set, and adjust the model parameters according to the test results to obtain the final Android malicious application detection model.

[0013] Step 6: Obtain the dynamic runtime features of the Android application to be tested, concatenate the dynamic runtime features and convert them into a grayscale image, input it into the final Android malicious application detection model, and obtain the detection results.

[0014] Furthermore, the reinforcement method in step 1 specifically includes the following steps:

[0015] Step 1-1: Construct a corresponding mapping table for the static code analysis features and dynamic runtime features of the APK file. When a feature exists, it is marked as 1, and when a feature does not exist, it is marked as 0. This will give you the mapping tables for the static code analysis features and dynamic runtime features respectively.

[0016] Step 1-1: Decompile the APK file and obtain the Application:name of the original APK file;

[0017] Steps 1-2: Prepare a shell DEX. Modify the Android:name field in the shell source code configuration file to the Application:name of the original APK file. The shell DEX is used to protect the original APK DEX. The task of decrypting and loading the original DEX is handed over to the shell DEX to prevent decompilation and analysis.

[0018] Steps 1-3: Save the original APK's DEX file to the end of the shell DEX file using AES encryption;

[0019] Steps 1-4: Replace the original DEX file in the original APK file with the shell DEX file, and modify the applicationandroid:name field in AndroidManifest.xml in the original APK file to enable booting from the shell DEX file;

[0020] Steps 1-5: Re-sign and repackage the APK processed in the previous step to obtain a reinforced signature.

[0021] Furthermore, the method for obtaining traffic data characteristics in step 2 includes the following steps:

[0022] Step 2-1: Install the sample APK file onto the Android virtual machine;

[0023] Step 2-2: Use ADB Shell Monkey to simulate multiple random interactions between the user and the corresponding installed APP;

[0024] Steps 2-3: Simulate user interaction by using the Scapy library to acquire traffic. During the extraction process, HTTP, TCP, and UDP packets are filtered out and saved as files, which are the traffic data characteristics of the sample.

[0025] Furthermore, the method for obtaining dynamic runtime features is to use AndroPyTool to generate dynamic runtime feature files corresponding to the sample APK files.

[0026] Furthermore, step 3 includes the following steps:

[0027] Step 3-1: Concatenate the traffic characteristics with the binary bit streams corresponding to the dynamic runtime characteristics to obtain a fused feature representation;

[0028] Step 3-2: Map the fused feature representation to the intensity value range of the grayscale image through frequency linear mapping;

[0029] Step 3-3: After fusing traffic features and dynamic runtime features, a frequency mapping strategy is used to express the distribution of different features. The resulting grayscale image presents the superposition effect of traffic features and dynamic runtime features. The horizontal dimension of the image corresponds to the sequential arrangement of feature data, while the vertical dimension corresponds to the amplitude or intensity of the features.

[0030] Furthermore, the deep learning classification model in step 4 is an EfficientNet model with an added attention mechanism.

[0031] Compared with the prior art, the present invention, by adopting the above technical solution, has the following advantages:

[0032] (1) This invention designs and implements an automated batch hardening method, which realizes batch hardening of Android applications by dynamically loading the original DEX using the shell DexClassLoader;

[0033] (2) This invention also implements an automated batch traffic acquisition tool, AutoTrafficTool, to acquire the traffic data generated by each sample during runtime. Simultaneously, a dynamic analysis program is used to acquire dynamic runtime feature data.

[0034] (3) A method for detecting malicious applications that can resist code hardening is proposed. Based on fuzz testing and network traffic packet extraction, the network traffic features of the target sample are automatically captured; simultaneously, the dynamic runtime features of the target application are obtained by monitoring the corresponding runtime function calls and system status. The extracted two types of features are then concatenated and mixed in context and converted into a grayscale image for detection based on a deep learning model.

[0035] (4) The deep learning model used in this invention is EfficientNet v2 and Attention. Based on the EfficientNet v2 deep learning model and the introduction of the Attention mechanism, significant results have been achieved in model lightweighting and performance optimization. Through careful design of network structure, parameter pruning, and quantization techniques, the computational and storage resource requirements of the model are reduced while maintaining high performance, making it suitable for deployment in resource-constrained environments. Utilizing the multi-scale feature learning capability of EfficientNet v2, the model's ability to capture malware samples of different sizes and complexities is enhanced. This helps improve the robustness of the model, making it more adaptable to malware variants and emerging threats. By introducing the Attention mechanism, the model can focus more on key features, increasing its attention to malware behavior. This mechanism helps improve the accuracy of the model, reduce the false positive rate, and thus enhance the reliability of malware detection.

[0036] (5) Since the model and training of the present invention are designed for hardened malware, the detection results will not be affected by hardening. The present invention uses dynamic runtime features and traffic features to improve the robustness of detection methods that use only static features or single features.

[0037] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Attached Figure Description

[0038] Figure 1 This is a flowchart of a robust detection method for Android malicious applications based on hybrid features, according to the present invention.

[0039] Figure 2 Flowchart for hardening APKAutoSheller;

[0040] Figure 3 Flowchart for automating traffic acquisition using AutoTrafficTool;

[0041] Figure 4 This is a network structure diagram of the deep learning models EfficientNet v2 and Attention. Detailed Implementation

[0042] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0043] This invention provides a robust detection method for Android malicious applications based on hybrid features. Under the background of security protection, the traffic characteristics and dynamic runtime characteristics of Android application software are mixed into a grayscale image. A deep learning framework is used to train the deep learning model EfficientNet v2 and Attention to obtain the Android malware detection model of this invention, which predicts whether the Android application software belongs to malware.

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions used in this invention will be described clearly and completely below with reference to the accompanying drawings. The examples given are only for explaining this invention and are not intended to limit the scope of this invention.

[0045] This invention provides a robust detection method for Android malicious applications based on hybrid features, the flowchart of which is shown below. Figure 1 As shown, it includes:

[0046] S1: Collect datasets of malware and benign software;

[0047] In step S1, the present invention uses two publicly available datasets: the CICMalDroid 2020 dataset built by the Canadian Cyber ​​Security Institute (CIC) and the Drebin dataset from the Institute for Systems Security at the Technical University of Brunswick. The CICMalDroid 2020 dataset contains more than 17,341 Android samples, including VirustoTalService, Contagio Security Blog, AMD, Maldozer, and other datasets used in recent research contributions; the Drebin dataset contains 5,560 malicious samples.

[0048] like Figure 2 As shown, S2: The parsed APK unpacking file is obtained by decompiling, and each sample in the dataset is automatically hardened and signed to form a new hardened dataset.

[0049] Step S2A-1 utilizes apktool and apksigner, both tools included in the Android SDK Build Tool. apktool can completely extract resource, dex, manifest, and xml files from an APK installation package; it can also modify resource files and rebuild an APK. apksigner is a tool used to digitally sign APKs during Android application development. Digital signing is a crucial step in the Android application distribution process, ensuring the integrity and authenticity of the APK. When an APK is digitally signed, it means that the APK's content has not been altered since the signing and also indicates the identity of the signer.

[0050] Step S2A-2, to obtain and modify AndroidManifest.xml and classes.dex, you first need to unpack the APK using apktool. After modification, repackage it using apktool, and finally re-sign the repackaged APK file. To ensure that apktool can properly decompile and recompile the APK, AndroidManifest.xml must be modified first, and then the DEX file replaced. If the DEX is replaced first, the encrypted data will be lost when recompiling the APK, resulting in package errors. In this step, apktool is mainly used to decompile the APK, modify AndroidManifest.xml through code, and then recompile to regenerate a new APK. In this way, the application's functionality is transparent to the user, but it is more difficult for unauthorized users to access it. For hardened shells that require advance preparation, the AndroidManifest.xml configuration file is included. This file is primarily used to guide the decryption and loading of the encrypted original DEX file. AES encryption is used, and to prevent key brute-force attacks, a random UUID is used as the key. The original DEX file is encrypted, and the key is saved within it. Finally, the encrypted original DEX file is appended to the end of the shell DEX file, which then replaces the original APK. During application runtime, the hardened shell accesses the key storage area, retrieves the key, and dynamically decrypts the application's code and resources, enabling the application to run normally.

[0051] like Figure 3 As shown, S3: Obtain the traffic data generated during the runtime of each sample.

[0052] Before acquiring traffic data, an Android virtual machine was deployed. A batch of APK samples were placed in the installation folder, and the APKs were installed onto the Android virtual machine using the Android Debug Bridge (ADB) install command. ADB Shell Monkey was used to simulate 100 random interactions between the user and the corresponding installed app. This process was performed almost simultaneously with step two, as many swipes and clicks during user interaction generate network traffic data. The Scapy library was used for traffic acquisition, filtering out HTTP, TCP, and UDP packets during the extraction process. Finally, the data was saved to a file. The current app was uninstalled using ADB uninstall. The strategy of installing one APK file at a time, waiting for it to finish running, uninstalling it, and then installing the next one was adopted to avoid data packet corruption caused by installing or running multiple apps simultaneously. The latest version of AndroPyTool was downloaded from the official AndroPyTool repository or another specified location. The APK files generated in step 3 were placed in a suitable directory. AndroPyTool will generate a dynamic runtime feature file in JSON format. The generated file was then viewed to obtain the analysis results.

[0053] S4: Convert the traffic feature .pcap file and the dynamic runtime feature .json file into binary bit streams and store them as byte sequences. Then concatenate the two files and convert them into a mixed grayscale image.

[0054] First, the two binary bitstreams are concatenated sequentially to create a new fused feature representation. Then, a frequency-linear mapping is used to map the merged data to the intensity range of a grayscale image. Frequency mapping is a mapping strategy commonly used to convert feature data into image or color representations to highlight the frequency or importance of features. This strategy utilizes the frequency distribution information of the data to determine the allocation of pixel values ​​or colors, thereby visualizing the frequency of feature occurrences in an image. After fusing traffic features and dynamic runtime features, the frequency mapping strategy can effectively express the distribution of different features. The final grayscale image presents the superposition effect of traffic features and dynamic runtime features, where the horizontal dimension of the image corresponds to the sequential arrangement of feature data, and the vertical dimension corresponds to the amplitude or intensity of the features. This data fusion and visualization method may help capture potential correlations between features in the image domain and may provide a novel perspective for further analysis and recognition.

[0055] A frequency mapping method is used to normalize the byte frequency of the input data and place it into a predefined 256*256 NumPy array initialized with all zeros to convert it into a grayscale image. Specifically, feature frequencies are calculated, and the frequency or occurrence count of each feature value is calculated statistically from the merged feature data. Based on the feature frequency information, different grayscale levels can be used to represent features of different frequencies. Higher frequencies are mapped to brighter grayscale values, while lower frequencies are mapped to darker grayscale values. The frequency information of the feature data is mapped to pixel values ​​to generate a frequency-mapped image representation. In the grayscale image, frequently occurring features will appear brighter.

[0056] like Figure 4 As shown, S5: Build the network structure model of EfficientNet v2 and Attention.

[0057] An Attention mechanism is introduced, which allows the model to focus more on important features. In this step, by defining an Attention layer, the model can more flexibly adjust its focus on the input during the learning process. Using EfficientNet v2 as the base model, a custom Attention layer is added to its output, enabling the model to better capture important information during learning and inference. The model structure includes a global average pooling layer, a fully connected layer, a Dropout layer, and an output layer. The shape of the input image is defined, typically choosing an appropriate size based on the design of EfficientNet. The number of output classes is also defined to suit the specific classification task. The model is compiled using an appropriate optimizer, loss function, and evaluation metric to prepare it for training. A summary of the model is printed to view its hierarchical structure and the number of parameters.

[0058] EfficientNet v2 uses scaling in both depth and width to design the model:

[0059]

[0060]

[0061] Where d is the depth and w is the width. α is the scaling factor of the model, and β are the hyperparameters of the depth and width of the base model.

[0062] The attention mechanism focuses the model's attention on different parts by assigning attention to weights. In the simplified self-attention mechanism, given the input sequence X = (x1, x2, ..., x...),... n The weights are calculated as follows:

[0063]

[0064] Where, f(x) i ) is about the input x i This is a certain representation. In this way, the sum of the attention weights is 1, ensuring global attention.

[0065] The final output of the model can be calculated via forward propagation, where f model (x) represents the model's prediction function, and θ is the model's parameters:

[0066] Output = f model (x;θ)

[0067] Here, x is the input sample. In classification problems, the softmax function is typically used to obtain the probability distribution for each class:

[0068]

[0069] Where, y i It is the probability that the sample belongs to the i-th category.

[0070] These formulas provide a mathematical description of some key concepts in the EfficientNet v2 and Attention models.

[0071] EfficientNet v2's lightweight design results in superior computational and parameter efficiency, making it suitable for resource-constrained devices and applications. The introduction of an Attention mechanism enhances the model's learning and utilization of key features, contributing to improved performance on complex data. Through multi-scale feature learning using EfficientNet v2, the model better adapts to inputs of varying scales and complexities, enhancing its ability to detect variant malware. The model integrates the Attention mechanism with the features of EfficientNet v2, considering security throughout the entire model lifecycle, including training, deployment, and updates. The Attention mechanism improves the model's interpretability, allowing users to better understand the model's focus on input data, facilitating debugging and optimization. The model's design considers adaptability to different environments and the need for continuous learning, enabling it to dynamically adjust and update in the face of new threats, maintaining high efficiency.

[0072] S6: Input the extracted fusion call sequence features into the client model.

[0073] The features processed in step S5 are used to train the model in EfficientNet v2 and the Attention model deep learning model. This invention optimizes and adjusts the parameters on EfficientNet v2, and the final model parameters are shown in Table 1. After comparing experiments with different parameters, it was found that for grayscale images converted from mixed features of dynamic runtime features and traffic features, the SGD optimization algorithm is used, with a cross-entropy loss function of 32 per batch, a learning rate of 0.001, and 100 training epochs.

[0074] Table 1 Training parameters of EfficientNet v2 network

[0075]

[0076] To make the simulated scenarios in the dataset of this invention more realistic, the extracted features are randomly divided into an 8:2 training set and a test set. During training, the dataset features are then randomly shuffled before being uploaded for training. After multiple rounds of training, the gradient of the client-side model is obtained.

[0077] S7: After multiple rounds of iterative training, the Android malware detection model of this invention is finally obtained. The evaluation metrics of this invention are F1, Acc, Pre, and Recall. The specific formulas are described as follows:

[0078]

[0079]

[0080]

[0081]

[0082] TP: Represents the number of samples that were actually positive but were predicted as positive; FP: Represents the number of samples that were actually negative but were predicted as positive; FN: Represents the number of samples that were actually positive but were predicted as negative; TN: Represents the number of samples that were actually negative but were predicted as negative. Additionally, TP+FP represents the total number of samples predicted as positive, and similarly, FN+TN represents the total number of samples predicted as negative. TP+FN represents the total number of samples that were actually positive, and FP+TN represents the total number of samples that were actually negative.

[0083] Accuracy is the proportion of correctly classified samples out of the total number of samples, and it is the simplest and most intuitive evaluation metric for classification problems. Precision reflects the model's ability to distinguish negative samples; the higher the precision, the stronger the model's ability to distinguish negative samples. Recall reflects the model's ability to identify positive samples; the higher the recall, the stronger the model's ability to identify positive samples. The F1 score is a combination of both; the higher the F1 score, the more robust the model.

[0084] The method provided by this invention has the following advantages or beneficial technical effects:

[0085] This invention proposes a robust detection method for hardened Android malicious applications based on hybrid features. Static reverse engineering has significant limitations for hardened software, and traditional static detection methods struggle to detect hardened malware samples. This paper proposes a method combining traffic data features and dynamic runtime features. This method fuses traffic features and dynamic runtime features, converts them into grayscale images, and then uses deep learning to classify the malware grayscale images. This provides a new method and approach for future research in the field of Android malware.

[0086] The above description provides examples of the preferred embodiments of the present invention, and any parts not described in detail are common knowledge to those skilled in the art. The scope of protection of the present invention is determined by the claims, and any equivalent modifications made based on the technical teachings of the present invention are also within the scope of protection of the present invention.

Claims

1. A method for robust detection of hardened Android malicious applications based on hybrid features, characterized in that, The method comprises the following steps: Step 1, a data set containing a plurality of benign and malicious Android applications is established, the data set is hardened and signed, and is divided into a training set, a verification set and a test set, wherein the training set and the verification set are used for training, and the test set is used for testing; The hardening method in step 1 specifically comprises the following steps: Step 1-1, the static code analysis features and dynamic runtime features of the APK file are constructed into corresponding mapping tables, and when a certain feature exists, it is marked as 1, and when a certain feature does not exist, it is marked as 0, to obtain the mapping tables corresponding to the static code analysis features and the dynamic runtime features respectively; Step 1-1, the Application: name of the original APK file is obtained by decompiling the APK file; Step 1-2, a shell DEX is prepared, the Android: name field in the shell source code is modified to the Application: name of the original APK file, the shell DEX is used to protect the DEX of the original APK, the tasks of decryption and loading of the original DEX are handed over to the shell DEX, and anti-decompilation analysis is prevented; Step 1-3, the DEX file of the original APK is saved to the tail of the shell DEX through AES encryption; Step 1-4, the original DEX file in the original APK file is replaced with the shell DEX, and the application android: name field in the AndroidManifest.xml of the original APK file is modified, so that the shell DEX is started; Step 1-5, the APK processed in the step is re-signed and packaged to obtain a hardened signature; Step 2, dynamic runtime features and traffic data features of samples in the data set are acquired; Step 3, the traffic features and the dynamic runtime features of each sample are spliced to form a fusion calling sequence feature, and are converted into a grayscale image; The step 3 comprises the following steps: Step 3-1, the binary bit streams corresponding to the traffic features and the dynamic runtime features are connected to obtain a fusion feature representation; Step 3-2, the fusion feature representation is mapped into the intensity value range of the grayscale image through frequency linear mapping; Step 3-3, after the traffic features and the dynamic runtime features are fused, a frequency mapping strategy is adopted to express the distribution of different features, and finally a grayscale image is generated, which presents the superposition effect of the traffic features and the dynamic runtime features, wherein the horizontal dimension of the image corresponds to the sequential arrangement of the feature data, and the vertical dimension corresponds to the amplitude or intensity of the feature; Step 4, a deep learning classification model is constructed, the deep learning classification model integrates a self-defined attention mechanism, the output of the deep learning classification model is the classification category of the input data, the grayscale image corresponding to the sample in the training set is input into the deep learning classification model, the deep learning classification model is trained, and the verification set is used for verification, so as to obtain a trained classification model; Step 5, the trained classification model obtained in the step is tested by using the test set, model parameters are adjusted according to the test result, and finally an Android malicious application detection model is obtained; Step 6, obtain the dynamic runtime features of the to-be-tested Android application, splice the dynamic runtime features, and convert them into a grayscale image, input the final Android malicious application detection model, and obtain a detection result. 2.The method of claim 1, wherein the method further comprises: The method for obtaining the traffic data features in step 2 comprises the following steps: Step 2-1, install the sample APK file on an Android virtual machine; Step 2-2, simulate random interaction between a user and the corresponding installed APP multiple times using ADB Shell Monkey; Step 2-3, during the interaction of the user, use the Scapy library to obtain traffic, filter out HTTP, TCP and UDP data packets during the extraction process, and save them as files, which are the traffic data features of the sample. 3.The method of claim 1, wherein the method further comprises: The method for obtaining dynamic runtime features is to use AndroPyTool to generate a dynamic runtime feature file corresponding to the sample APK file.

4. The method of claim 1, wherein the method is based on a hybrid feature. The deep learning classification model in step 4 is an EfficientNet model with an added attention mechanism.

Images