Big data based mobile application malicious component detection method

Abstract

The application provides a mobile application malicious component detection method based on big data, and in view of the challenge of diversified malicious code forms, a multi-element software heterogeneous behavior feature representation method with anti-confusion ability is proposed on the basis of different behavior features proposed in previous work. In view of the challenge of time dependence and lack of fine-grained malicious component detection, the application provides a mobile application malicious component detection technology based on big data. The application firstly collects existing mobile application information databases to construct corresponding malicious application and malicious code knowledge databases. Secondly, the application extracts program features and APP type knowledge to construct training samples. Finally, the application judges whether it is malicious software by using a deep learning algorithm, and further analyzes and detects malicious code blocks by using a multi-objective genetic algorithm.

Description

Technical Field

[0001] This invention relates to a method for detecting malicious components in mobile applications. Background Technology

[0002] In recent years, with the increasing popularity of mobile devices (smartphones, tablets, and in-vehicle mobile devices, etc.), more and more application developers have created mobile applications with more comprehensive functions to meet user needs. However, as mobile applications become increasingly integrated into people's lives, ensuring their security has become a hot topic in the industry. Mobile application risks bring a large number of uncertainties to social security, including threats such as remote control, economic and financial fraud, and privacy data breaches. Therefore, malware detection technology has attracted widespread attention from industry and academia.

[0003] Android is the most popular mobile operating system on the internet, but its open-source nature makes it a target for 97% of malware attacks. According to 360 Security's 2019 Mobile Security Report, 360 Security Brain intercepted approximately 1.809 million new mobile malware samples throughout 2019, averaging about 5,000 new mobile malware samples per day. McAfee's 2021 Mobile Threat Report indicates that cybercriminals attacked in 2019 by expanding and hiding mobile applications, making malware increasingly difficult to identify and eliminate. In 2020, as many vulnerabilities were closed or patched, these criminals continued to experiment with new lures and techniques to bypass security checks. For example, to evade detection, they submitted a clean version of their application during Google's review process and then introduced malicious code in subsequent updates. The ever-changing nature of malware weakens the effectiveness and versatility of detection tools. Furthermore, with increasingly sophisticated code obfuscation techniques, malware can significantly alter its code structure while retaining its original intent, reducing the density of malicious code and exhibiting a more complex and diverse trend. This presents new challenges for malware identification.

[0004] In addition, security analysts at major security companies face a large number of malware samples to be analyzed every day, which is a very time-consuming and labor-intensive task.

[0005] Currently, there are many machine learning-based Android malware detection methods that utilize features such as permissions, Android APIs, Android components, Dalvik code, and call graphs. DREBIN extracts features from the AndroidManifes.xml file and Dalvik bytecode through static analysis to detect malware. Extracted features include hardware information, permission requests, and application components. DeepRefiner has two malware detection layers; the second layer directly inputs the application's smali code into an LSTM-based model to detect malware applications, exhibiting robustness and high accuracy. ICCDetector uses ICC-related features to simulate application behavior. MaMaDroid abstracts method names and extracts call relationships to construct Markov chains to model application behavior. The Markov chains are then converted into feature vectors for classification. MalScan uses the centrality of sensitive Android APIs in the call graph as feature vectors for classification.

[0006] In general, machine learning-based methods first extract features from applications (e.g., Dalvik bytecode, graph structure, Android APIs, etc.) to model their behavior. Then, they train detection models using a large number of samples and use the trained models to detect malware applications. While these methods perform well on relevant datasets and have some generalization ability, they still face the following three challenges:

[0007] First, malware diversity: To evade existing Android malware detection systems, malware developers frequently modify malware source code to create multiple variants. However, most existing systems extract only single and simple pieces of information for behavioral modeling (e.g., API sequences, Android APIs, and graph structures), allowing malware to evade them.

[0008] Second, temporal dependencies: Temporal dependencies represent the execution logic of code, containing semantic and structural information (i.e., control flow, data flow, and call relationships). Capturing temporal dependencies is necessary because malicious components may be small parts of a malicious application (e.g., 2%), distributed among them. Most existing Android malware detection systems ignore it or only consider a portion of it, allowing malware to evade detection through specific modifications;

[0009] Third, there is a lack of fine-grained malicious component detection: Existing malicious code snippet detection systems can detect malicious code at the class or method level. However, their location results are determined by local information within a single unit (i.e., basic blocks, methods, and classes), but lack contextual information (such as adjacent methods), making it easy for them to detect benign parts as malicious. Furthermore, these methods detect a fixed number of malicious classes or methods and are not adaptive.

[0010] Therefore, in order to ensure the healthy development of the smart mobile device ecosystem, innovative technologies for detecting malicious components in Android mobile applications are urgently needed to solve the above challenges. Summary of the Invention

[0011] The technical problem this invention aims to solve is that existing Android malware systems struggle with three aspects: malware diversity, time dependence, and fine-grained detection of malicious components.

[0012] To address the aforementioned technical problems, the present invention provides a method for detecting malicious components in mobile applications based on big data, characterized by comprising the following steps:

[0013] Step 1: Extract ten semantic features from four dimensions of the mobile application to characterize its behavior, including: static permission request features and dynamic permission request features under the permission dimension; activity component features, broadcast component features, content provider component features and service component features under the component dimension; UI features under the resource dimension; and Android API function features, inter-component communication features and function call graph features under the code dimension.

[0014] Step 2: Based on the ten semantic features extracted in the four dimensions in Step 1, construct a function call graph CGJ with a sequence of method Jimple statement trees, denoted as G = {V, E}, where V represents the method node in the mobile application, E represents the call relationship between method nodes, and each abstract statement node v in V is represented as a Jimple statement tree and is characterized by a ten-dimensional vector. The value of each element in the ten-dimensional vector corresponds to the ten semantic features extracted.

[0015] Step 3: Convert the function call graph CGJ into a JST sequence JSTS;

[0016] Step 4: Input the JST sequence JSTS into an AST neural network with feature mapping based on ASTNN to obtain the malicious confidence of the input JST sequence JSTS, so as to distinguish between malicious features and benign features.

[0017] Step 5: For the JST sequence JSTS identified as malicious in Step 4, perform malicious component detection using a fine-grained malicious component detection algorithm based on a multi-objective genetic algorithm, further including the following steps:

[0018] Step 501: Select an initial subgraph from the function call graph CGJ as the initial population;

[0019] Step 502: Use a multi-objective genetic algorithm to detect malicious components in the JST sequence JSTS.

[0020] Preferably, in step 1, when obtaining the function call graph features, Spark pointers are used to analyze and mine the function call relationships in the mobile application, but the call relationships between API functions are excluded, thereby generating a function call graph, and the method sequence result is generated using the call relationship sequence generated at the same time;

[0021] In step 3, the JST sequences JSTS in one method are arranged in their statement order, and all the JST sequences JSTS are concatenated to generate the JST sequences JSTS of the function call graph CGJ based on the method sequence results.

[0022] Preferably, in step 2, the ten-dimensional vector is represented as [f1, f2, f3, ..., f10], where:

[0023] f1 corresponds to the static request permission feature, and its value represents the static request permission information of the abstract statement node v;

[0024] f2 represents the dynamic permission request feature, and its value indicates the dynamic permission request information of the abstract statement node v.

[0025] f3 to f6 correspond to the activity component characteristics, broadcast component characteristics, content provider component characteristics, and service component characteristics, and their values ​​indicate whether the abstract statement node v belongs to a method of a certain component class.

[0026] f7 corresponds to UI features, i.e., the UI components of the APP. Its value indicates whether the abstract statement node v belongs to a certain UI component or a callback method of a UI component: if the abstract statement node v belongs to a method of the UI class or contains a method call of the UI class, it means it belongs; otherwise, it means it does not belong.

[0027] f8 corresponds to the Android API function feature, and its value indicates whether the abstract statement node v calls the Android API function;

[0028] f9 corresponds to the inter-component communication feature, and its value indicates whether the abstract statement node v has inter-component communication behavior;

[0029] f10 corresponds to the function call graph feature, and its value indicates whether the abstract statement node v has been called.

[0030] The value range of each element is 1 or 0, where 1 indicates that the abstract statement node v has the feature, and 0 indicates that the abstract statement node v does not have the feature.

[0031] Preferably, step 501 further includes the following steps:

[0032] Step 5011: Convert the function call graph CGJ into an undirected graph G, using 0, 0.5 and 1 as three different edge weights to model Android API calls, one-way calls and two-way calls respectively. Among them, the weight 0 is set to exclude Android API calls.

[0033] Step 5012: Use Louvain's algorithm to find modules in the undirected graph G;

[0034] Step 5013: Map the found modules to the function call graph CGJ, which will serve as the initial subgraph for the subsequent multi-objective genetic algorithm.

[0035] Preferably, step 502 further includes the following steps:

[0036] Step 5021: Set the function call graph CGJ of the malicious mobile application as the initial search space Ms;

[0037] Step 5022: Based on the malicious subgraph obtained in the previous search, use a multi-objective genetic algorithm to search for malicious subgraphs in the current search space Ms;

[0038] Step 5023: Merge all solutions obtained in step 5022 with the malicious subgraph obtained in the previous round of search into a new malicious subgraph and assign it to the search space Ms. The updated search space Ms becomes the reduced search space Ms of the next round of multi-objective genetic algorithm. The search process continues until no solution better than the malicious subgraph obtained in the previous round of search is found in the current round of multi-objective genetic algorithm. Then the search process is terminated and the new search space Ms of the current round is used as the detected fine-grained malicious component.

[0039] Preferably, in step 5022, the following two stopping criteria are adopted:

[0040] Stopping Criterion 1) When the reverse of a first-layer individual is True and its age is 0, the search space shrinking stops, indicating that no newly generated malicious subgraph can dominate the malicious subgraph of the previous round in A iterations;

[0041] Stopping Criterion 2) When the reverse of a first-layer individual is False and its age is A, the search space is further reduced, indicating that there is a malicious subgraph that dominated the previous malicious subgraph in A iterations and survived in the first-layer F0.

[0042] Here, age is used to determine whether the genetic algorithm process should continue; individuals with True reverse are malicious subgraphs generated in the previous round of the genetic algorithm, while individuals with False reverse are malicious subgraphs generated in the current round.

[0043] Preferably, in step 502, the new method is introduced into a malicious subgraph or the old method is deleted from it by cross-simulating the recombination between modules of the multi-objective genetic algorithm through mutation of the multi-objective genetic algorithm.

[0044] Preferably, in step 5022, three fitness functions are designed: malicious confidence, Android API weight, and graph density, wherein:

[0045] Each Android API in the function call graph CGJ is assigned a weight representing its sensitivity, but only sensitive APIs used in the APIGRAPH are weighted. The number of normal permissions (NP) and dangerous permissions (DP) used by sensitive APIs (Cn and Cd) are calculated: if their numbers are both greater than 0, the weight of the normal permission (NPW) is set to Cd / Cn; otherwise, it is set to 0.5; the weight of the dangerous permission (DPW) is always set to 1. The weight of sensitive APIs is the sum of their permission weights. For APIs other than sensitive APIs, their weights are set to 0. The fitness function for Android API weights is defined as follows:

[0046]

[0047] W(S) evaluates the sum of weights of API-related edges in subgraph S, where the weight of each edge is equal to the weight of the sensitive API being called, and E(S) evaluates the number of edges in subgraph S.

[0048] Graph density represents the degree of edge connectivity in a subgraph S;

[0049] If a subgraph S has the highest objective score on one fitness function but lower scores on the other two fitness functions than other subgraphs, then it should not be placed in F0 based on preference.

[0050] Preferably, the crossover operation of the multi-objective genetic algorithm accepts two candidate individuals and generates two or one new offspring, using one of the following two methods:

[0051] Method 1 includes the following steps:

[0052] a) Merge the two candidate individuals into a larger graph and compute its maximum clique;

[0053] b) Randomly select two largest cliques as two bases;

[0054] c) Randomly shuffle these two bases and randomly select an adjacent clique from the remaining largest clique;

[0055] d) Combine the selected cliques with the currently traversed bases in a shuffled order to generate a new base;

[0056] e) Repeat steps c)-d) until all groups are combined, and the final two bases are the offspring;

[0057] Method 2 includes the following steps:

[0058] Only two candidate individuals are merged together.

[0059] Preferably, the mutation operation of the multi-objective genetic algorithm does not use the Android API for mutation, and includes addition and deletion methods, wherein:

[0060] For adding a method, randomly select one method from the candidate individuals and merge all adjacent methods;

[0061] For deleting methods, the process of adding methods is repeated, but a method is deleted or its adjacent methods are deleted with a fixed probability of 0.5.

[0062] To address the challenge of diverse malicious code forms, this invention, building upon previous work on different behavioral characteristics, proposes a multi-dimensional heterogeneous software behavioral feature representation method with anti-obfuscation capabilities. First, this invention extracts a multi-dimensional, multi-semantic set of software behaviors using static program analysis techniques. Then, it proposes a heterogeneous behavioral feature fusion method, employing feature selection and feature abstraction techniques to map heterogeneous behaviors into a homogeneous space, eliminating redundant information while preserving the core semantics of the software. This reduces the complexity of analysis while ensuring the behavioral features possess a certain degree of anti-obfuscation capability.

[0063] To address the challenges of time-dependent and lacking fine-grained malicious component detection, this invention proposes a mobile application malicious component detection technology based on big data. First, this invention constructs a corresponding knowledge database of malicious applications and malicious code by collecting information from existing mobile application (hereinafter referred to as "APP") program databases. Second, this invention constructs training samples by extracting program features and APP types. Finally, this invention uses deep learning algorithms to determine whether an application is malicious software and further utilizes a multi-objective genetic algorithm to analyze and detect malicious code blocks. Attached Figure Description

[0064] Figure 1 The AMCDroid system architecture is illustrated.

[0065] Figure 2 The ASTNNF neural network structure is illustrated.

[0066] Figure 3 This illustrates the malicious component detection algorithm MCD. Detailed Implementation

[0067] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0068] The mobile application malicious component detection system based on big data proposed in this invention (hereinafter referred to as the "AMCDroid detection system") can not only effectively detect malicious software, but also support fine-grained malicious component detection. For example... Figure 1 As shown, the AMCDroid detection system contains three components: (1) a heterogeneous feature behavior representation component (reads in the APK and represents it as a homogeneous graph and JSTS); (2) an ASTNNF component detection model (detects the malicious confidence of the input JSTS, used to output the malware detection results or as part of the malicious component detection component); and (3) a malicious component detection component (searches for malicious components in the homogeneous graph and outputs the detected malicious component subgraph).

[0069] 1. Heterogeneous Feature Behavior Representation Component

[0070] Current machine learning-based malware detection methods primarily model applications by extracting single-dimensional or single-semantic information from apps. Here, "dimensionality" represents coarse-grained information about the app (e.g., code dimension), while "semantics" represents fine-grained information (e.g., API functions used by the app). Consequently, existing methods struggle to capture all behavioral characteristics of an app. Therefore, this invention proposes a multi-dimensional, multi-semantic approach to modeling Android mobile application behavior.

[0071] Currently, due to the increasing development of technologies such as software obfuscation, and in response to the challenges of diversification and unknowns, this invention proposes a 4-dimensional, 10-semantic feature representation of APP behavior for comprehensive modeling. The specific features are shown in Table 1 below.

[0072]

[0073] Table 1. Multidimensional and multi-semantic feature types

[0074] Extracting static information from multiple dimensions and semantics enables more accurate modeling of app behavior, avoiding the loss of necessary information.

[0075] The four dimensions proposed in this invention are described in detail below:

[0076] 1) Permission Dimension: The Android permission system is one of the important mechanisms for ensuring device security. Since Android 6.0, permissions are divided into two categories—normal permissions (registered in the Manifest file) and dangerous permissions (registered in the Manifest file and dynamically requested at runtime). The Android permission system ensures that unauthorized apps cannot access user privacy data. However, users often don't pay attention to the permissions requested by apps when installing and using them, which gives malicious actors an opportunity. Malicious apps often request more permissions than benign apps. For example, many malicious apps dynamically request the SEND_SMS permission to send SMS messages and subscribe to advanced services. Therefore, permission information is extremely important for modeling and characterizing app behavior.

[0077] 2) Component Dimension: Android's four main components include Activity, BroadcastReceiver, Content Provider, and Service. These four components are registered in the Manifest file, with dynamic broadcasts able to be registered dynamically at runtime. From the Android Lifecycle perspective, the callback methods of these four components can be considered as the app's function entry points. Furthermore, the fully qualified class names of these four components often contain information about malicious apps, such as AirPush, Mecor, and Youmi. Therefore, Android's four main components are also considered as one source of extracted information.

[0078] 3) Resource Dimension: UI components are a crucial part of an app, forming a bridge between the user and the application. DeepIntent generates software by finding the mapping relationship between sensitive permission information in the app and the GUI.<icon,text,permissions> Malware classification and detection using triple sequences. This work demonstrates that GUI information can effectively characterize software intent and behavior.

[0079] 4) Code Dimension: Code is the core of an app; all software behaviors ultimately need to be executed through code. Therefore, the app code dimension can provide sufficient semantic information about the software. App code consists of Dalvik bytecode. After generating human-readable text through disassembly technology, it can be found that the data area structure of the dex file is mainly a sequential structure composed of class, method, and statement bytecode. Therefore, to more accurately represent software behavior, this invention introduces Android APIs, inter-component communication, and function call graphs to represent the code dimension and model software behavior.

[0080] The characteristics included in the code dimension are:

[0081] Android APIs: Android provides a large number of API functions to facilitate application development. These API functions allow applications to access information about the device, such as contact information, current location, battery level, and Wi-Fi connection status. Attackers can also exploit Android's API functions and users' lack of awareness regarding device permissions to steal private data or perform other malicious operations.

[0082] Inter-Component Communication (ICC): Communication between components within an app is accomplished using Intents. Intents are commonly used to launch components and pass data between them. Intents are divided into explicit Intents and implicit Intents. Explicit Intents require the recipient to be explicitly specified, while implicit Intents do not. Malware can indirectly communicate with specific components by setting actions and categories as the recipient, and exploit Intents to obtain sensitive data, leading to risks such as data leaks, Intent hijacking, impersonation, and phishing applications.

[0083] Function call graph (hereinafter referred to as "CG"): In Java, a function call graph represents the calling relationship between methods, where the caller is called the Caller and the callee is called the Callee. The calling relationship characterizes the software's behavioral intent and contains rich semantics. Different methods can be combined to accomplish a specific function, and the method name often directly indicates the method's behavior; for example, sendTextMessage() means sending an SMS message. Furthermore, the function call graph is key to heterogeneous fusion of 10 features. The AMCDroid detection system uses Spark pointers to analyze and mine function call relationships in apps, but excludes API call relationships. This is because APIs are considered benign methods by default, and this also helps reduce wasted analysis and detection time.

[0084] Since the features extracted in this invention come from multiple different dimensions, and each dimension has a different representation, this invention uses a graph to integrate heterogeneous behavioral features into a unified feature form to reduce the complexity of subsequent work and retain sufficient semantic information. This invention constructs a function call graph (hereinafter referred to as "CGJ") with a sequence of method Jimple statement trees (hereinafter referred to as "JST"), G = {V, E}. In the function call graph G, V represents a method node in the APP, and E represents the call relationship between methods. For example, if V1 is the caller of V2, the call relationship between V1 and V2 can be represented as: Caller(V1) invoke Callee(V2). For each abstract statement node v in V, its representation is a JST, and a 10-dimensional vector [f1, f2, f3, ..., f10] is used to represent this abstract statement node. Each element in the 10-dimensional vector corresponds to one of the 10 features in Table 1, as described in detail below:

[0085] f1 corresponds to the first feature in Table 1, namely static request permission, and its value represents the static request permission information of the abstract statement node v.

[0086] f2 corresponds to the second feature in Table 1, namely dynamic permission request, and its value represents the dynamic permission request information of the abstract statement node v.

[0087] f3-f6 correspond to the third to sixth features in Table 1, namely the four major components of the APP. Their values ​​indicate whether the abstract statement node v belongs to the method of a certain component class.

[0088] f7 corresponds to the seventh feature in Table 1, namely the UI component of the APP. Its value indicates whether the abstract statement node v belongs to a certain UI component or the callback method of the UI component: if the abstract statement node v belongs to a method of the UI class or contains a method call of the UI class, it means it belongs; otherwise, it means it does not belong.

[0089] f8 corresponds to the eighth feature in Table 1, namely Android APIs, and its value indicates whether the abstract statement node v calls an Android API function;

[0090] f9 corresponds to the ninth feature in Table 1, namely Inter-Component Communication (ICC), and its value indicates whether the abstract statement node v has inter-component communication behavior;

[0091] f10 corresponds to the tenth feature in Table 1, namely function call, and its value indicates whether the abstract statement node v has been called.

[0092] Each element has a value of either 1 or 0, where 1 indicates that the abstract statement node v possesses the characteristic, and 0 indicates that the abstract statement node v does not possess the characteristic. Based on this, an application is represented using a graph.

[0093] To capture time dependency information in CGJ, we convert it into a JST sequence (hereinafter referred to as "JSTS"). To ensure that the CGJ JSTS contains some information about the method execution sequence, we utilize Spark pointer analysis to generate the CG and use the concurrently generated call relationship sequence to generate the method sequence result (hereinafter referred to as "MSR"). We generate the MSR by sequentially traversing the call relationships. The caller is traversed before the callee in a call relationship, and a method is traversed only once. The JSTS within a method are arranged in the order of their statements, and all JSTS are concatenated to generate the CGJ JSTS based on the MSR.

[0094] 2. Component Detection Model ASTNNF

[0095] The component detection model is an AST neural network with feature mapping based on ASTNN (ASTNNF), which predicts the malicious confidence of the input. It is used in malware detection and malicious component detection.

[0096] ASTNNF is built on top of the ASTNN model and can capture information between the feature vectors of JST. There are three reasons for using ASTNN as the basis: 1) it has achieved state-of-the-art classification results in other related fields; 2) its encoding layer provides efficient computation for multi-way trees; and 3) RNN has a strong ability to capture time dependencies.

[0097] Figure 2The network structure of ASTNNF is shown. ASTNNF contains four layers similar to ASTNN: 1) JSTS Input Layer; 2) Encoding Layer; 3) Bidirectional GRU Layer; and 4) Max Pooling Layer. The encoding layer differs from ASTNN. JSTS is first encoded into an encoding vector ei by an RvNN-based sentence encoder. Furthermore, ASTNNF's encoding layer adds an extra connection operation between the JSTS feature vector and the corresponding ei, unlike ASTNN. The final encoding vector contains the lexical and syntactic information of JSTS, along with their feature vectors. This final encoding vector is fed into the bidirectional GRU to capture temporal dependencies (semantic and structural information, i.e., control flow, data flow, and call relationships) between JSTs from both directions (forward and backward). Finally, ASTNNF stacks the hidden states hi generated in the previous stage, utilizes max pooling to capture their core features, and generates a representation of the input JSTS.

[0098] The function of ASTNNF is to distinguish between malicious and benign features. Therefore, we use a linear layer and Softmax as our classifier, combined with ASTNNF, and use cross-entropy as our loss function.

[0099] The output is the malicious confidence score of the input JSTS. It is the probability that the JSTS is malicious and is ∈ [0,1]. In malware detection, a malicious confidence score exceeding a certain threshold (e.g., 0.5) indicates that the input JSTS is malicious; otherwise, the JSTS is benign.

[0100] 3. Malicious Component Detection Component

[0101] Malware detection is crucial for activating further detection, as benign applications lack malicious behavior. Existing methods detect malicious components by selecting the top n classes or methods with the highest malicious confidence as the outcome. However, no one can know the true malicious application in advance. An inappropriate n for a specific malicious application can generate a large number of false positives or false negatives. To adaptively detect malicious components, we propose a fine-grained malicious component detection algorithm, MCD, based on a multi-objective genetic algorithm (GA), on CGJ. Furthermore, to accurately search for solutions, we introduce community detection to select the initial population, reducing the loss of initial evolutionary information. MCD consists of two main parts: 1) selecting an initial subgraph from CGJ as the initial population; 2) utilizing GA to detect malicious components.

[0102] 1) Initial subgraph selection

[0103] The choice of initial subgraphs (individuals) impacts the detection of malicious components. Small initial individuals (e.g., a single method) lack contextual information, which can affect the evolutionary direction of the genetic algorithm, as malicious behavior often consists of multiple methods (i.e., call chains). Large initial individuals (e.g., CGJ) contain sufficient contextual information, but are difficult to refine. To address this, we utilize modules (also known as communities) within the graph as initial subgraphs. From a software engineering perspective, each module should be highly modular. A module in the program can be viewed as a function (feature) of the user, and malicious behavior is distributed across these functions. Therefore, this reduces the impact on the evolutionary direction and makes it easier to refine the solution. Furthermore, multi-objective genetic algorithms require modules to avoid approximating a single objective and to prevent the loss of initial evolutionary information (e.g., small initial individuals may lack graph structure information).

[0104] Specifically, we use community detection to find modules. First, we transform CGJ into an undirected graph G. To make the undirected graph G contain more original call relationships, we use 0, 0.5, and 1 as three different edge weights to model Android API calls, one-way calls, and two-way calls, respectively. We set a weight of 0 to exclude Android API calls because many methods can call the same API, which would lead to confusion during the community detection process. Then, we use the Louvain algorithm to find modules in the undirected graph G. The Louvain algorithm is suitable for large graphs with limited computational resources and is well-suited for undirected graphs G. Finally, the found modules are mapped to CGJ as the initial subgraph for the subsequent genetic algorithm process.

[0105] 2) Detect malicious components

[0106] Fine-grained malicious components can be distributed across one or more modules. We utilize MOSA for heuristic detection of CGJ because it tends to favor potentially better solutions. Crossover in a genetic algorithm can simulate recombination between modules, while mutation can introduce new methods into a subgraph or remove old methods. However, crossover and mutation are difficult to refine for solutions containing too many methods. To address this issue, we introduce a strategy to narrow the search space for the genetic algorithm. Therefore, the malicious component detection (MCD) disclosed in this invention comprises the following three steps:

[0107] Step 1: Locate the module in the search space;

[0108] Step 2: Evolutionary detection of malicious subgraphs through crossover and mutation;

[0109] Step 3: Narrow down the search space and repeat steps 1 and 2 above until fine-grained malicious components are obtained.

[0110] The MCD algorithm is attached. Figure 3 The above is an explanation and theoretical description of what was given in the text.

[0111] The input to MCD is the malicious application's CGJ. MCD first initializes the parameters and variables used in the genetic algorithm process and sets CGJ as the initial search space Ms. Figure 3 In the first line of the code shown, PC is the crossover probability, PI is the probability of choosing a crossover, PM is the mutation probability, M is the number of offspring, T is the maximum number of iterations, and A is the maximum age for contraction or stopping.

[0112] Next, MCD calls the GAPROCESS method to search for a malicious subgraph in the current search space Ms, where the parameter s of GAPROCESS is the malicious subgraph obtained in the previous search.

[0113] Then, MCD merges all solutions output by the GAPROCESS method (the currently detected malicious subgraph) with the malicious subgraph obtained in the previous round of search into a new subgraph and assigns it to the search space Ms. The updated search space Ms becomes the narrowed search space for the next round of the genetic algorithm process. The next round of the genetic algorithm is necessary because the new search space Ms may still contain some benign methods. However, if no solution better than the malicious subgraph s obtained in the previous round is found in the current round of the genetic algorithm, f is set to False to terminate the search process, and the new search space Ms of the current round is used as the detected fine-grained malicious component.

[0114] The GAPROCESS method described above aims to search for malicious subgraphs within a given search space. The input to GAPROCESS is a search space Ms and malicious subgraphs s obtained in the previous round. MCD first selects an initial subgraph using the Louvain algorithm and initializes the population pop (lines 11 and 12) and other variables. Note that the subgraphs generated in the current round have the same initialization in line 11. MCD iteratively searches for a solution based on MOSA. However, GAPROCESS also includes a newer PREFERENCE-SORTING-U method (described in the fitness function section), and two additional stopping criteria: Stopping criterion 1) stops narrowing the search space when a first-layer individual's reverse is True and its age is 0 (line 19); Stopping criterion 2) continues narrowing the search space when a first-layer individual's reverse is False and its age is A (line 20). age is used to determine whether the genetic algorithm process should continue and also ensures fine-grained processing. Individuals with Truereverse are malicious subgraphs generated in the previous round of the genetic algorithm, while those with Falsereverse are malicious subgraphs generated in the current round. Stopping criterion one indicates that no newly generated malicious subgraph dominates the previous subgraph within A iterations. Therefore, it is highly probable that a malicious component has been found. Stopping criterion two indicates that a malicious subgraph dominates the previous subgraph within A iterations and survives in the first layer F0. Therefore, the search space should be reduced and a new genetic algorithm process should be initiated. Furthermore, exceeding T iterations also indicates that a malicious component has been found. MCD assigns all solutions in the first layer F0 of the initial population pop to s and returns s and f. We treat the solutions in the first layer F0 as equally important to reduce the loss of potentially malicious methods.

[0115] The theoretical basis of the algorithm is described below:

[0116] Encoding: The search space (CGJ or its subgraphs) is encoded as a two-dimensional graph structure containing call relationships, providing rich information for the genetic algorithm.

[0117] Fitness Functions: We designed three fitness functions (FFs): Malicious Confidence (MC), Android API Weight (AW), and Graph Density (GD). MC represents the probability of malicious activity calculated using ASTNNF (AST Neural Network-based Fuzzing). However, since ASTNNF may not directly learn specific malicious behaviors meaningful to humans, we introduce AW to identify meaningful and sensitive behaviors through sensitive APIs (Application Programming Interfaces). Furthermore, malicious behaviors often appear in the form of call chains, so GD ensures that local methods within a component are as interconnected as possible. Therefore, the higher the values ​​of all three fitness functions, the more likely the component is to be malicious.

[0118] Malicious Confidence-ASTNNF is used to evaluate MC, and the fitness function is defined as follows:

[0119] MC0 = 1 - A b (J)

[0120] Where J represents the input JSTS of the subgraph, A b This is a function that evaluates the benign confidence of the input, with values ​​ranging from [0,1]. The traversal result of the subgraph is generated by taking the corresponding methods in the MSR of CGJ in order to produce its JSTS.

[0121] In Android API Weighting - CGJ, each Android API is assigned a weight representing its sensitivity. We only set weights for sensitive APIs (SAs) used in APIGRAPH. To obtain the weights, the AMCDroid detection system calculates the number of normal permissions (NP) and dangerous permissions (DP) used by the SAs, Cn and Cd: if both are greater than 0, the weight of the normal permission (NPW) is set to Cd / Cn; otherwise, it is set to 0.5; the weight of the dangerous permission (DPW) is always set to 1. This definition ensures a balance between NPW and DPW. Therefore, the weight of an SA is the sum of its permission weights. For other APIs, their weights are set to 0. Finally, the fitness function of AW is defined as follows:

[0122]

[0123] W(S) evaluates the sum of weights of API-related edges in subgraph S, where the weight of each edge is equal to the weight of the sensitive API being called, and E(S) evaluates the number of edges in subgraph S.

[0124] Graph density (GD) represents the density of a graph, i.e., the degree of edge connectivity, and is defined as follows:

[0125]

[0126] Where n and m represent the number of nodes and edges in the subgraph S, respectively.

[0127] Furthermore, in line 17 of the algorithm, if a subgraph S has the highest objective score on one fitness function but lower scores on the other two fitness functions, below other subgraphs, then it should not be placed in F0 based on preference. For example, a benign subgraph might have the highest AW score and the lowest MC and GD scores. Therefore, we reverse the PREFERENCE-SORTING method in MOSA, storing these subgraphs in a special F0.5 and storing the remaining subgraphs starting from F0. The new PREFERENCE-SORTING-U method favors subgraphs with more balanced three objective scores, potentially containing more complete malicious behavior.

[0128] Crossover and mutation: Since the encoded individuals exist in the form of a graph structure, two update operations are required to generate diverse offspring and take into account calling relationships.

[0129] Crossover—The new crossover operation accepts two candidate individuals and generates two or one new offspring, corresponding to two versions. For version 1, it includes the following steps:

[0130] a) MCD merges two candidate individuals into a larger graph and calculates its maximum clique.

[0131] b) Randomly select two largest cliques as two bases.

[0132] c) Randomly shuffle the two bases and randomly select an adjacent clique (sharing a common vertex) from the remaining largest clique.

[0133] d) Combine the selected cliques with the currently traversed bases in a shuffled order to generate a new base.

[0134] e) Repeat the above process (c)-d)) until all groups are combined, and the final two bases are the offspring.

[0135] Version 1 takes into account the randomness and calling relationships in the graph.

[0136] For version 2, MCD merges only two candidate individuals together, which provides the opportunity to reconstruct a larger subgraph and generate a child. The selection of the two versions is controlled by PI.

[0137] Mutation—The new mutation operations include adding and deleting methods. For adding a method, MCD randomly selects a method from a candidate graph and merges all adjacent methods (i.e., connected methods). For deleting a method, MCD repeats the above process, but deletes a method or both the method and its adjacent methods with a fixed probability of 0.5. This new mutation operation guarantees the relevance of the added or deleted method to the subgraph. It is worth noting that MCD does not select Android APIs for mutation because they are connected to methods in many different modules.

[0138] We implemented AMCDroid using PyTorch, Androguard, Python 3.8, Java 1.8, IC3, and FlowDroid, with approximately 10,000 lines of code. The CUDA version is 11.4. The server we used contained an Intel(R) Core(TM) i9-10900X CPU, 128GB of RAM, and an RTX 3080 (10G) graphics card.

Claims

1. A method for detecting malicious components in mobile applications based on big data, characterized in that, Includes the following steps: Step 1: Extract ten semantic features from four dimensions of the mobile application to characterize its behavior, including: static permission request features and dynamic permission request features under the permission dimension; activity component features, broadcast component features, content provider component features and service component features under the component dimension; UI features under the resource dimension; and Android API function features, inter-component communication features and function call graph features under the code dimension. Step 2: Based on the ten semantic features extracted in the four dimensions in Step 1, construct a function call graph CGJ with a sequence of method Jimple statement trees, denoted as G = {V, E}, where V represents the method node in the mobile application, E represents the call relationship between method nodes, and each abstract statement node v in V is represented as a Jimple statement tree and is characterized by a ten-dimensional vector. The value of each element in the ten-dimensional vector corresponds to the ten semantic features extracted. Step 3: Convert the function call graph CGJ into a JST sequence JSTS; Step 4: Input the JST sequence JSTS into an AST neural network with feature mapping based on ASTNN to obtain the malicious confidence of the input JST sequence JSTS, so as to distinguish between malicious features and benign features. Step 5: For the JST sequence JSTS identified as malicious in Step 4, perform malicious component detection using a fine-grained malicious component detection algorithm based on a multi-objective genetic algorithm, further including the following steps: Step 501: Select an initial subgraph from the function call graph CGJ as the initial population; Step 502: Use a multi-objective genetic algorithm to detect malicious components in the JST sequence JSTS.

2. The method for detecting malicious components in mobile applications based on big data as described in claim 1, characterized in that, In step 1, when obtaining the function call graph features, Spark pointers are used to analyze and mine the function call relationships in the mobile application, but the call relationships between API functions are excluded, thereby generating a function call graph, and the method sequence result is generated using the call relationship sequence generated at the same time; In step 3, the JST sequences JSTS in one method are arranged in their statement order, and all the JST sequences JSTS are concatenated to generate the JST sequences JSTS of the function call graph CGJ based on the method sequence results.

3. The method for detecting malicious components in mobile applications based on big data as described in claim 1, characterized in that, In step 2, the ten-dimensional vector is represented as [f1, f2, f3, ..., f10], where: f1 corresponds to the static request permission feature, and its value represents the static request permission information of the abstract statement node v; f2 corresponds to the dynamic request permission feature, and its value represents the dynamic request permission information of the abstract statement node v. f3 to f6 correspond to the activity component characteristics, broadcast component characteristics, content provider component characteristics, and service component characteristics, and their values ​​indicate whether the abstract statement node v belongs to a method of a certain component class. f7 corresponds to UI features, i.e., the UI components of the APP. Its value indicates whether the abstract statement node v belongs to a certain UI component or a callback method of a UI component: if the abstract statement node v belongs to a method of the UI class or contains a method call of the UI class, it means it belongs; otherwise, it means it does not belong. f8 corresponds to the Android API function feature, and its value indicates whether the abstract statement node v calls the Android API function; f9 corresponds to the inter-component communication feature, and its value indicates whether the abstract statement node v has inter-component communication behavior; f10 corresponds to the function call graph feature, and its value indicates whether the abstract statement node v has performed a function call. The value range of each element is 1 or 0, where 1 indicates that the abstract statement node v has the feature, and 0 indicates that the abstract statement node v does not have the feature.

4. The method for detecting malicious components in mobile applications based on big data as described in claim 1, characterized in that, Step 501 further includes the following steps: Step 5011: Convert the function call graph CGJ into an undirected graph G, using 0, 0.5 and 1 as three different edge weights to model Android API calls, one-way calls and two-way calls respectively. Among them, the weight 0 is set to exclude Android API calls. Step 5012: Use Louvain's algorithm to find modules in the undirected graph G; Step 5013: Map the found modules to the function call graph CGJ, which will serve as the initial subgraph for the subsequent multi-objective genetic algorithm.

5. The method for detecting malicious components in mobile applications based on big data as described in claim 1, characterized in that, Step 502 further includes the following steps: Step 5021: Set the function call graph CGJ of the malicious mobile application as the initial search space Ms; Step 5022: Based on the malicious subgraph obtained in the previous search, use a multi-objective genetic algorithm to search for malicious subgraphs in the current search space Ms; Step 5023: Merge all solutions obtained in step 5022 with the malicious subgraph obtained in the previous round of search into a new malicious subgraph and assign it to the search space Ms. The updated search space Ms becomes the reduced search space Ms of the next round of multi-objective genetic algorithm. The search process continues until no solution better than the malicious subgraph obtained in the previous round of search is found in the current round of multi-objective genetic algorithm. Then the search process is terminated and the new search space Ms of the current round is used as the detected fine-grained malicious component.

6. The method for detecting malicious components in mobile applications based on big data as described in claim 1, characterized in that, In step 5022, the following two stopping criteria were adopted: Stopping Criterion 1) When the reverse of a first-layer individual is True and its age is 0, the search space shrinking stops, indicating that no newly generated malicious subgraph can dominate the malicious subgraph of the previous round in A iterations; Stopping Criterion 2) When the reverse of a first-layer individual is False and its age is A, the search space is further reduced, indicating that there is a malicious subgraph that dominated the previous malicious subgraph in A iterations and survived in the first-layer F0. Here, age is used to determine whether the genetic algorithm process should continue; individuals with True reverse are malicious subgraphs generated in the previous round of the genetic algorithm, while individuals with False reverse are malicious subgraphs generated in the current round.

7. The method for detecting malicious components in mobile applications based on big data as described in claim 1, characterized in that, In step 502, the new method is introduced into a malicious subgraph or the old method is deleted from it by crossover simulation between modules of the multi-objective genetic algorithm and mutation of the multi-objective genetic algorithm.

8. The method for detecting malicious components in mobile applications based on big data as described in claim 1, characterized in that, In step 5022, three fitness functions are designed: malicious confidence, Android API weight, and graph density, where: Each Android API in the function call graph CGJ is assigned a weight representing its sensitivity, but only sensitive APIs used in the APIGRAPH are weighted. The number of normal permissions (NP) and dangerous permissions (DP) used by sensitive APIs (Cn and Cd) are calculated: if their numbers are both greater than 0, the weight of the normal permission (NPW) is set to Cd / Cn; otherwise, it is set to 0.5; the weight of the dangerous permission (DPW) is always set to 1. The weight of sensitive APIs is the sum of their permission weights. For APIs other than sensitive APIs, their weights are set to 0. The fitness function for Android API weights is defined as follows: W(S) evaluates the sum of weights of API-related edges in subgraph S, where the weight of each edge is equal to the weight of the sensitive API being called, and E(S) evaluates the number of edges in subgraph S. Graph density represents the degree of edge connectivity in a subgraph S; If a subgraph S has the highest objective score on one fitness function but lower scores on the other two fitness functions than other subgraphs, then it should not be placed in F0 based on preference.

9. The method for detecting malicious components in mobile applications based on big data as described in claim 1, characterized in that, The crossover operation of the multi-objective genetic algorithm accepts two candidate individuals and generates two or one new offspring, using one of the following two methods: Method 1 includes the following steps: a) Merge the two candidate individuals into a larger graph and compute its maximum clique; b) Randomly select two largest cliques as two bases; c) Randomly shuffle these two bases and randomly select an adjacent clique from the remaining largest clique; d) Combine the selected cliques with the currently traversed bases in a shuffled order to generate a new base; e) Repeat steps c)-d) until all groups are combined, and the final two bases are the offspring; Method 2 includes the following steps: Only two candidate individuals are merged together.

10. The method for detecting malicious components in mobile applications based on big data as described in claim 1, characterized in that, The mutation operation of the multi-objective genetic algorithm does not select the Android API for mutation, including the add and delete methods, wherein: For adding a method, randomly select one method from the candidate individuals and merge all adjacent methods; For deleting methods, the process of adding methods is repeated, but a method is deleted or its adjacent methods are deleted with a fixed probability of 0.5.

Images