A Mobile Malicious App Detection Method Based on Incremental Learning

Abstract

This invention discloses a mobile malicious app detection method based on incremental learning. The invention includes static analysis, feature generation, model construction, and detection steps. This invention extracts features that can fully represent the behavior of malicious apps, such as permission features required to access private data, APIs for acquiring and transmitting data, and sensitive API pairs that better characterize behavior. These features can effectively represent the intent and behavior of the app and have high distinguishability between malicious and normal apps. Therefore, using these features can significantly improve the effectiveness of detection. Simultaneously, this invention addresses the problem that most existing machine learning-based methods use batch processing to train models, making them difficult to adapt to detecting sudden outbreaks of specific types of malicious apps.

Description

Technical Field

[0001] This invention discloses a method for detecting malicious mobile apps based on incremental learning. Background Technology

[0002] With the widespread adoption of smartphones, the number of apps providing various convenient services to users has exploded. At the same time, some malicious mobile apps have spread by taking advantage of this popularity, posing a significant threat to user privacy. Privacy breaches can range from disrupting a user's daily life to seriously endangering their personal safety. Therefore, identifying and detecting malicious apps to protect user privacy is crucial. Currently popular methods for detecting malicious mobile apps mainly fall into three categories: static detection, dynamic detection, and machine learning methods based on the former two.

[0003] Static analysis of malicious apps typically involves decompiling the app's source code or executable file, analyzing the code, and identifying potential malicious behaviors. This method doesn't require real-time program execution to observe and record app behavior, resulting in very high detection efficiency and code coverage, making it suitable for large-scale app detection. If an app's abstract model is established, static analysis can traverse all paths of the abstract model built from intermediate code, and its execution speed is fast. However, static analysis of decompiled app code struggles to handle malicious apps employing obfuscation, mapping, or hardening techniques. Many malicious apps use these detection evasion techniques to modify their behavior, making it difficult for static analysis to construct or incorrectly construct the app's abstract model.

[0004] Unlike static detection methods, dynamic methods collect information recorded by an app during its dynamic execution, analyze the collected app behavior, and detect whether the app has performed malicious actions. Therefore, dynamic detection methods must be executed on a virtual or real device. Since dynamic detection technology can only collect data after the program has executed, it can ensure the authenticity of the collected program execution information and reduce the false positive rate of malware detection. Obfuscated code and hardened software need to be executed after decompression, so dynamic detection technology can solve these problems to some extent. On the other hand, dynamic detection technology also has its own limitations. Since it can only collect information after the program has executed, some malware will check whether it is running in a virtual detection environment. If it detects that it is running in a virtual environment, it will not perform malicious actions. Therefore, dynamic detection technology suffers from poor coverage and low execution efficiency.

[0005] Malware and legitimate software have different purposes, resulting in different execution behaviors. To achieve these different objectives, they request different permissions, and their API usage reflects these different behaviors. From this perspective, machine learning methods can naturally be used to distinguish between malware and legitimate software. Malware often follows specific rules when performing malicious actions, such as using specific APIs (sources) to obtain sensitive information and using other specific APIs (sinks) to transmit information, or executing malicious code only under certain system event triggers. These behavioral rules often differ from those of legitimate software. Using machine learning methods to distinguish between malware and legitimate software requires a dataset containing both malware and legitimate software, with each sample having a clear label. Experienced security researchers then need to extract sufficiently discriminative features, such as permissions or sensitive APIs. Using these defined features, each sample forms a feature vector, which is then used to train a machine learning model to detect unknown samples. The effectiveness of machine learning detection methods is closely related to the manually constructed feature set; extracting highly discriminative information as features can significantly improve detection efficiency and accuracy.

[0006] However, most existing machine learning-based methods train models in batch processing. In reality, the spread and discovery of malicious apps are time-series, and batch processing models often struggle to adapt to sudden outbreaks of specific types of malicious apps. Summary of the Invention

[0007] Based on the problems of current malicious mobile app detection methods mentioned in the background section, this invention proposes a mobile malicious app detection method based on incremental learning.

[0008] The mobile malicious APP detection method proposed in this invention includes three steps: 1) static analysis step; 2) feature generation step; 3) model construction and detection step.

[0009] 1) Static analysis steps

[0010] Collect app samples labeled with both benevolent and malicious intent, and preprocess each app.

[0011] Use static analysis tools to decompress the APP package and obtain all the files contained in the APP, including Manifest.xml and class.dex files;

[0012] The extracted class.dex file is decompiled to obtain the class file for each class created in the app.

[0013] 2) Feature generation steps

[0014] Extract the permissions requested in the Manifest.xml file;

[0015] Iterate through all intermediate code in the app, extract sensitive APIs, and analyze whether the class file contains multiple sensitive APIs. If it does, extract them in the form of sensitive API pairs.

[0016] Principal component analysis was performed on the collected training sample set to reduce the dimensionality of sensitive APIs;

[0017] The extracted permissions, sensitive APIs, and reduced-dimensional sensitive API pairs are concatenated to form a feature vector for the app.

[0018] 3) Model construction and detection steps.

[0019] Using the acquired training samples, an online incremental ensemble classifier is trained.

[0020] The unlabeled APP samples collected subsequently were detected and identified.

[0021] The beneficial effects of this invention are as follows: This invention extracts features that can fully represent the behavior of malicious apps, such as permission features required to access private data, APIs for acquiring and transmitting data, and sensitive API pairs that are more representative of behavior. These features can effectively characterize the intent and behavior of an app and have a high degree of differentiation between malicious and normal apps. Therefore, using these features can significantly improve the effectiveness of detection. At the same time, this invention can solve the problem that most existing machine learning-based methods use batch processing to train models, making them difficult to adapt to detecting sudden outbreaks of specific types of malicious apps. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the schematic diagram of the method proposed in the present invention. The described embodiments are some embodiments of the present invention, but not all embodiments.

[0024] Reference Figure 1 In this paper, a mobile malicious APP detection method based on incremental learning includes three steps: 1) static analysis step; 2) feature generation step; 3) model construction and detection step.

[0025] In this invention, the static analysis step involves collecting APP samples labeled as benevolent and malicious, and preprocessing each APP. First, a static analysis tool is used to decompress the APP package, obtaining all files contained within the APP, including the Manifest.xml file. Then, the decompressed class.dex file is decompiled to obtain the class files for each class created within the APP.

[0026] In this invention, the feature generation steps first extract the requested permissions from the Manifest.xml file. Then, all intermediate code in the app is traversed to extract sensitive APIs. The class files are analyzed to determine if multiple sensitive APIs are present; if so, they are extracted as pairs of sensitive APIs. Finally, since the feature vectors of sensitive API pairs have high dimensionality, principal component analysis is performed on the collected training sample set to reduce their dimensionality. For each sample, the extracted permissions, sensitive APIs, and the dimensionality-reduced sensitive API pairs are concatenated to form a feature vector for the app.

[0027] In this invention, the model building and detection steps are as follows: First, a newly created online incremental ensemble classifier is trained using training samples obtained from the feature generation step; then, APP samples with unknown labels collected subsequently are detected and identified; finally, the model is updated using APP samples with labels further confirmed by experts, thereby improving the model's ability to detect currently popular APPs.

[0028] The following is an example:

[0029] Step 1: Collect malicious mobile apps and normal apps as a dataset and preprocess them.

[0030] Step 1.1 uses three real-world malicious mobile app datasets. Two are widely used Android malicious app benchmark datasets, Genome and Drebin. The third dataset consists of over 20,000 malware samples downloaded from AndroZoo, a platform that collects both malicious and benign Android apps. These malicious datasets are then mixed with over 20,000 commonly used datasets obtained from Google Play to form a larger training dataset.

[0031] Step 1.2: For each APP sample, use a static analysis tool to decompress the sample and extract the Manifest.xml and class.dex files.

[0032] Step 1.3: Decompile the class.dex file to obtain Jimple intermediate code.

[0033] Step 2: Extract features and generate feature vectors.

[0034] Step 2.1: Extract permission features from the Manifest.xml obtained in Step 1. Additionally, iterate through the Jimple intermediate code to extract two types of sensitive APIs: 1) APIs for retrieving data (source); 2) APIs for transmitting data (sink). Use the frequency of occurrence of the extracted sensitive APIs as features.

[0035] Step 2.2: Using Java classes as the scope, extract sensitive API pairs within the same scope. Since there are many types of sensitive API pairs, in order to reduce the feature dimensions created, retain the selected high-frequency sensitive API pairs.

[0036] Step 2.3: The obtained permission features, sensitive API features, and high-frequency sensitive API features are numerically processed and concatenated into a feature vector.

[0037] Step 3: Build an ensemble classifier model based on incremental learning, train it using the dataset, and detect unknown apps to determine whether they are malware.

[0038] Step 3.1: This embodiment designs an ensemble classifier model based on incremental learning. The ensemble classifier model has multiple base classifiers of the same type. At the beginning of training, multiple data blocks containing different samples are used to train the base classifiers separately. After collecting a portion of currently popular malicious apps and obtaining their labels, the ensemble classifier's detection capability for the current apps is tested first. If the detection performance is significantly reduced, the ensemble classifier is pruned, removing the worst-performing base classifier, and a new base classifier is trained using the current samples and added to the ensemble classifier.

[0039] Step 3.2: Use an incrementally updated ensemble classifier to detect the currently acquired unknown mobile apps.

Claims

1. A method for detecting malicious mobile apps based on incremental learning, characterized in that... Includes the following steps: 1) Static analysis steps Collect app samples labeled with both benevolent and malicious intent, and preprocess each app. Use static analysis tools to decompress the APP package and obtain all the files contained in the APP, including Manifest.xml and class.dex files; The extracted class.dex file is decompiled to obtain the class file for each class created in the app; 2) Feature generation steps Extract the permissions requested in the Manifest.xml file; Iterate through all intermediate code in the app, extract sensitive APIs, and analyze whether the class file contains multiple sensitive APIs. If it does, extract them in the form of sensitive API pairs. Principal component analysis was performed on the collected training sample set to reduce the dimensionality of sensitive APIs; The extracted permissions, sensitive APIs, and reduced-dimensional sensitive API pairs are concatenated to form a feature vector of the APP. 3) Model construction and detection steps; Using the acquired training samples, an online incremental ensemble classifier is trained. The ensemble classifier model has multiple base classifiers of the same type. At the start of training, multiple data blocks containing different samples are used to train the base classifiers separately. After collecting a portion of currently popular malicious apps and obtaining their labels, the ensemble classifier's ability to detect the current apps is tested first. If the detection performance is significantly reduced, the ensemble classifier is pruned, the worst-performing base classifier is removed, a new base classifier is trained using the current samples, and then added to the ensemble classifier. The app samples with unknown tags collected subsequently were detected and identified.

2. The mobile malicious APP detection method based on incremental learning according to claim 1, characterized in that: The sensitive APIs mentioned in step 2) include APIs for obtaining data and APIs for transmitting data.

3. The mobile malicious APP detection method based on incremental learning according to claim 1, characterized in that: The sensitive APIs mentioned in step 2) come from the same region, which is defined by the Java class.

4. The mobile malicious APP detection method based on incremental learning according to claim 1, characterized in that: Step 3) also includes updating the model using APP samples with labels further confirmed by experts to improve the model's ability to detect currently popular APPs.

Images