Android malicious software detection method based on multi-task learning mechanism

Abstract

The invention provides an Android malicious software detection method based on a multi-task learning mechanism, and belongs to the technical field of mobile security and malicious software detection. Comprising the steps of 1, extracting malicious family correlation, and extracting correlation among different malicious families through a multi-task learning mechanism by adopting a deep learning model based on a CNN (convolutional neural network) and a Bi-LSTM (bidirectional long short-term memory) network; 2, generating a malicious space, and depicting a high-risk sample region through feature space generation, threat region identification and region priority ranking; 3, identifying high-risk samples, and screening the samples based on region priorities and position attributes; and step 4, decision optimization: carrying out fine adjustment on the detection model by combining high-risk samples and known sample anchor points so as to improve the detection capability of unknown malicious families. According to the method, through multi-task learning and malicious space analysis, the detection effect of the method on unknown Android malicious software is improved.

Description

Technical Field

[0001] This invention relates to mobile security and malware detection technology, specifically to an Android malware detection method based on a multi-task learning mechanism. Background Technology

[0002] In recent years, Android has become the world's mainstream mobile operating system due to its openness, and its application ecosystem continues to expand. According to third-party statistics, Android devices accounted for over 71% of the global market share in 2025, with over 2.65 million applications. However, this open ecosystem has also brought severe security challenges. Research shows that the number of malware is exploding, with an average of over 9,000 new malicious samples added daily. Attackers often use techniques such as code obfuscation and repackaging to tamper with legitimate applications and inject malicious code to achieve unauthorized operations such as data theft and data consumption. However, traditional detection methods suffer a significant drop in efficiency and accuracy when faced with large-scale and diverse malware variants. Furthermore, existing static signature-based detection schemes lack adaptability to unknown threats and family variants, posing a challenge to the deployment of Android malware detection solutions.

[0003] Currently, detection methods based on behavioral analysis and machine learning have achieved some success. The basic principle of these methods is to train statistical models for classification tasks by extracting static features (such as permissions, API call sequences) or dynamic features (such as runtime behavior) of the application. However, such methods rely heavily on the scale and quality of the training data. This can lead to insufficient generalization ability of the trained models due to the sparsity of the training data. Furthermore, while deep learning models improve detection performance by automatically learning feature representations, they may fail to detect attacks with few samples or from unknown malicious attack families due to the lack of learned behavioral patterns.

[0004] In contrast, active learning mechanisms alleviate data dependency issues by iteratively selecting high-value samples to update the model. This approach constructs a sample selection strategy, prioritizing the labeling of samples that contribute most to model optimization to gradually adjust the decision boundary and adapt to new malware family variants. While this alleviates the false positive problem of existing methods for unknown malware, there is still room for research on how to improve the value of sample selection. Based on this, this invention proposes an Android malware detection method based on a multi-task learning mechanism. The core idea of ​​this method is to construct a correlation representation between malware families through multi-task joint training, using this as the basis for improving the generalization of the detection model. Subsequently, a sample priority evaluation mechanism based on distance similarity generates an efficient sample selection sequence, aiming to achieve stable performance and rapid adaptability of the detection model in data-scarce scenarios, providing an effective solution to cope with the ever-evolving malware threat. Summary of the Invention

[0005] To address the issues of poor performance of existing Android malware detection methods in detecting unknown malware families in sparse data scenarios and the inefficiency of expert-driven sample screening mechanisms, this invention proposes a malware detection method (FreezeMal) based on multi-task learning.

[0006] This invention provides an Android malware detection method based on a multi-task learning mechanism, comprising:

[0007] Step 1: Extract the correlation between malware families. A deep learning model based on convolutional neural network (CNN) and bidirectional long short-term memory network (Bi-LSTM) is used to extract the correlation between different malware families through a multi-task learning strategy.

[0008] Step 2: Generate a malicious space by characterizing high-risk sample regions through feature space generation, threat region identification, and region priority ranking.

[0009] Step 3, high-risk sample identification, involves sorting and selecting samples based on regional priority and sample location;

[0010] Step 4, Decision Optimization: By combining high-risk samples and known sample anchors, the detection model is fine-tuned to improve the detection capability against unknown malicious families.

[0011] Firstly, the specific steps of step 1 above are as follows:

[0012] Step 1.1: Design the embedding module, combining CNN and dual Bi-LSTM for feature dimensionality reduction and representation learning;

[0013] Step 1.2: Construct a multi-task branch structure, with each branch corresponding to a binary classification task of a malware family. Through joint optimization, improve the model's ability to model the differences and commonalities between families.

[0014] Secondly, the specific steps of step 2 above are as follows:

[0015] Step 2.1: Use the trained embedding layer to map the original features to a low-dimensional space to construct a malicious behavior space;

[0016] Step 2.2: Identify high-risk areas located at the decision boundary based on the cluster centers and radii of known malware families.

[0017] Thirdly, the specific steps of step 3 above are as follows:

[0018] Step 3.1: Calculate the deviation and sample density of each high-risk area to determine the sampling priority of the area;

[0019] Step 3.2: Sort the samples based on their distance from the center of the region, and use a truncated normal distribution to allocate the number of samples, giving priority to samples at the decision boundary.

[0020] Fourthly, the specific steps of step 4 above are as follows:

[0021] Step 4.1: Integrate the multi-task branch structure into a unified classification module, and retain the embedding layer as a feature extractor;

[0022] Step 4.2: Fine-tune by combining high-risk samples with some known samples, and use residual structure and Dropout layer to prevent overfitting and knowledge forgetting.

[0023] Fifthly, the aforementioned risk area identification mechanism specifically includes:

[0024] Risk area identification: Based on the clustering structure of known malware families in the embedding space, identify high-risk areas located at decision boundaries between families or with dense distribution. Let... and The sample clusters are from two different families, and their sample sizes are respectively and The cluster radii are respectively and Then the distance between the risk areas of the two families and cluster a is calculated by formula (1), where This represents a vector representing the centers of two clusters. Then, based on the distance... Formula (2) is used to sequentially combine known families to obtain the vector pointing set of different risk areas. .in This is a union operation, where n is the number of family clusters. Let be the center vector corresponding to family cluster a. Let be the center vector corresponding to family cluster b.

[0025] (1);

[0026] (2);

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] 1. This invention offers significant advantages in malware family association modeling. Its core lies in jointly optimizing multiple family classification tasks through a multi-task learning mechanism. This design enables the embedding space to effectively capture the commonalities and differences in behavior among families, enhancing the model's robustness in data-sparse scenarios. Furthermore, the proposed embedding module combines CNN and Bi-LSTM, simultaneously extracting local patterns and sequence dependencies, thereby improving the separability and discriminability of feature representations.

[0029] 2. This invention proactively explores potential threats by constructing a malicious behavior space and identifying high-risk areas. It prioritizes regions based on geometric features and sample density, efficiently filtering out representative high-risk samples. Furthermore, this scheme employs a distance-based sampling strategy to recommend high-quality samples, reducing the time cost of manual screening. Attached Figure Description

[0030] Figure 1 This is a flowchart of an Android malware detection method based on a multi-task learning mechanism.

[0031] Figure 2 This is the deep learning model used in the multi-task learning mechanism of this invention.

[0032] Figure 3 This is a comparison of FreezeMal with learning-based benchmark methods in detecting time-drift datasets; (a) is the distribution of the balanced accuracy metric, (b) is the distribution of the precision metric, (c) is the distribution of the recall metric, and (d) is the distribution of the F1 score metric.

[0033] Figure 4 This is a comparison of FreezeMal with learning-based benchmark methods in detecting unknown malicious families; (a) is a distribution of the balanced accuracy metric, (b) is a distribution of the precision metric, (c) is a distribution of the recall metric, and (d) is a distribution of the F1 score metric.

[0034] Figure 5 This is an analysis of the impact of the number of malicious families in the training set on FreezeMal.

[0035] Figure 6 The results are the evaluation results of FreezeMal under different high-risk sample learning rates; (a) shows the performance of the four metrics on Airpush, (b) shows the performance of the four metrics on Dowgin, (c) shows the performance of the four metrics on Fakeinst, (d) shows the performance of the four metrics on Kuguo, (e) shows the performance of the four metrics on Smsreg, (f) shows the performance of the four metrics on Smssend, (g) shows the performance of the four metrics on Youmi, and (h) shows the average performance of the four metrics. Detailed Implementation

[0036] The following is in conjunction with the appendix Figures 1-6 The present invention will be further illustrated by the embodiments described herein. It should be noted that the embodiments described are intended only to facilitate the understanding of the present invention and are not intended to limit it in any way.

[0037] This invention aims to address the problem of unknown malicious family attacks in Android malware detection by proposing a detection method based on a multi-task learning mechanism. The invention provides a comprehensive framework for malicious behavior feature embedding, risk area identification, and detection strategy optimization, and extensive experiments have demonstrated the feasibility and effectiveness of the method.

[0038] like Figure 1 As shown, the present invention provides an Android malware detection method based on a multi-task learning mechanism, comprising:

[0039] Step 201, Malware family correlation extraction, extracts the correlation between different malware families from the original features through a multi-task learning mechanism.

[0040] The APK file is parsed to obtain Dalvik-style code. Regular expressions are used to match the invoke directive in the code, and system functions are further determined based on the package information of the target function.

[0041] The manifest configuration file of the APK file is parsed to extract the permission information requested by the APK file;

[0042] For the four major Android components, by performing class hierarchy analysis on the APK file and determining the implementation classes of Activity, Service, BroadcastReceiver and ContentProvider included in the system based on the inheritance relationship;

[0043] Feature fusion and vectorization combine the extracted permissions, API call sequences, component information, etc., into a unified feature representation;

[0044] The single sample label of each sample is converted into multiple binary classification labels required for multi-task learning;

[0045] By constructing a hybrid embedding module that includes CNN and Bi-LSTM, the model hyperparameters are trained based on a multi-task learning mechanism.

[0046] Step 2011, Label Conversion: Reconstruct the original single-label classification problem into a multi-family detection problem. For a scenario containing K malicious families and 1 benign family, define K+1 binary classification tasks and convert the one-hot encoded labels into two-dimensional task labels.

[0047] Step 2012, Multi-task learning modeling: Constructing such as Figure 2The diagram illustrates a deep learning architecture comprising a shared embedding module and multiple task-specific branches. The embedding module employs a hybrid CNN and Bi-LSTM structure, where two CNN layers (16 channels, 3×3 convolutional kernels) extract spatial features, and a Bi-LSTM (16 hidden units) captures temporal dependencies. Finally, the model is projected onto a two-dimensional space through a linear transformation layer. Both hybrid structures utilize existing general structural models, except for the aforementioned features. The task branches employ a unified architecture: a Dropout layer with a dropout rate of 0.2, a 16-unit fully connected layer, and a Softmax output layer.

[0048] Step 2013, Hyperparameter Tuning: A class-specific margin adjustment strategy based on frequency-based parameters is adopted as shown in formula (3). The original prediction loss is used for the first 60% of the training cycle, and class-specific margins are introduced later. Through scaling factor The probability distribution is sharpened, and finally, a binary cross-entropy loss optimization model is applied.

[0049] (3);

[0050] in, The output layer signal quantity obtained for each training sample, The output layer signal quantity of the initial training samples. is the trainable marginal parameter, e is the natural logarithm, s is the scaling factor, j is the sample class index, t is the training batch, and τ is the proportion of the training batch that needs to be adjusted.

[0051] Step 202, Malicious Behavior Space Construction: Based on the output features of the embedding layer, a malicious behavior space is constructed to identify risk areas.

[0052] Step 2021, Feature Space Generation: The embedding layer of multi-task learning is used to reduce the dimensionality of the original features, which alleviates the problem of mixed benign and malicious samples in the original feature space and improves the clustering effect of samples of the same family.

[0053] Step 2022, High-risk area identification: Based on the geometric characteristics (centroid and radius) of known family clusters, risk areas with low similarity to known families are identified by calculating the central regions between clusters.

[0054] Step 2023, Regional Priority Ranking: Calculate the deviation score for each risk region using formulas (4) and (5). and density fraction The deviation score is based on the cosine of the angle between the region and its nearest family cluster, while the density score is based on... The number of samples within the radius should prioritize regions with high scores in both areas.

[0055] (4);

[0056] in Let i be the center point of the i-th risk area. As the center of the benevolent sample cluster, It is a cosine function. Angle As the center of the malicious family cluster, It is a central collection of malware family clusters. For separation The latest malicious family cluster center.

[0057] (5);

[0058] in For a single sample point, For the candidate point set, As the center point of the risk area, This is the distance radius threshold.

[0059] Step 203, risk sample identification, sorting and selecting samples based on regional priority and sample distance.

[0060] Step 2031: Using the Gaussian kernel function, the samples are sorted according to their distance from the center of the risk area; Instance number determination: The number of samples allocated to each region is determined using a truncated normal distribution as shown in formula (6), where the distribution parameters... Standard deviation upper boundary parameters Lower boundary parameter ( (Number of risk areas).

[0061] (6);

[0062] Step 2032, Sample selection: Unknown samples within the same cluster are sorted using a distance-based decay strategy as shown in formula (7), and the sample count is calculated. Relative to the regional center The priority, of which Sampling is performed based on sample priority, and repeated 5 times to enhance randomness.

[0063] (7);

[0064] in To select the probability, For sample points, Center of the sample cluster region The normalization constant is It is an exponential function. The standard deviation is denoted as .

[0065] Step 204, Decision Optimization: By introducing residual blocks and combining high-risk samples and known sample anchors, the detection model is fine-tuned to improve the detection capability of unknown malware.

[0066] Step 2041: By integrating multi-task branches into a unified classification module, a deep learning model containing residual blocks is constructed. To prevent knowledge forgetting, this stage includes both high-risk samples and randomly selected known training samples for model fine-tuning.

[0067] Step 2402: The branch layers are integrated into a unified classification module. The output of the embedding layer is used as input, and the module passes through a Dropout layer with a dropout rate of 0.2 and three residual blocks (32×48 hidden neurons). Finally, the label probability is output through a Softmax layer.

[0068] This invention primarily evaluates the proposed FreezeMal method, using three publicly available datasets for performance testing. Table 1 shows the distribution of the datasets used in the experiments, including CICmalDroid2020, VirusTotal, and TCAMD. CICmalDroid2020 contains 11,598 APK files, with a positive sample ratio of 84.52%; VirusTotal contains 48,412 APK files, with a positive sample ratio of 50%; and TCAMD contains 105,911 APK files, with a positive sample ratio of 11.75%.

[0069] Table 1 shows the dataset selected for the experimental testing phase of this invention:

[0070] ;

[0071] To verify the effectiveness of the multi-task learning-based detection method proposed in this invention, we conducted evaluation experiments from different perspectives. The evaluation metrics used were balanced accuracy, precision, recall, and F1 score, as shown in formulas (8)-(11).

[0072] (8);

[0073] (9);

[0074] (10);

[0075] (11);

[0076] True positives (TP) represent the number of correctly classified positive samples, true negatives (TN) represent the number of correctly classified negative samples, false positives (FP) correspond to the number of negative samples that are misclassified as positive samples, and false negatives (FN) represent the number of positive samples that are misclassified as negative samples.

[0077] Table 2 shows the evaluation results of FreezeMal on a single dataset. In the intra-family evaluation scenario, the training and test sets were randomly split in an 8:2 ratio. In the cross-family evaluation scenario, 80% of malicious families were selected as the training set, and the remaining 20% ​​as the test set for model evaluation. FreezeMal achieved an average F1 score of 98.18% in the intra-family detection scenario, and maintained high detection performance in the cross-family evaluation scenario, with an average F1 score of 96.89%.

[0078] Table 2 shows the evaluation results of FreezeMal on a single dataset:

[0079] ;

[0080] at the same time, Figure 3 and Figure 4 This study presents a comparison with five other benchmark detection methods in scenarios involving concept drift and detection of unknown malicious families. Observing the experimental results in these two categories reveals that FreezeMal demonstrates superior performance in F1 scores compared to other baseline methods.

[0081] Furthermore, the experiment also analyzed in depth the key factors affecting the method's performance. In addition to comparing the final detection results, this invention also verified the effectiveness of the proposed multi-task learning embedding and risk sample fine-tuning mechanism. The experimental results are shown in Table 3 and... Figure 5 As shown in the experimental results on family diversity and the number of risk samples, the multi-task learning mechanism proposed in this invention can effectively enhance the model's generalization ability. While keeping the total number of training samples constant, the detection performance shows a stable upward trend when the number of training families increases from 2 to 6.

[0082] Table 3 shows the performance analysis of FreezeMal under multi-task learning mechanism and model fine-tuning mechanism:

[0083] ;

[0084] Finally, we further explored the effect of different learning rates on FreezeMal, and the experimental results are as follows: Figure 6 As shown in the figure. The results indicate that the risk sample selection mechanism can reliably screen high-value samples, achieving significant performance improvement when the active sampling ratio is 1%-2%.

[0085] The above experimental results demonstrate the effectiveness of this invention in dealing with unknown malware and concept drift.

[0086] This invention constructs a malicious behavior space through multi-task learning, identifies risk areas based on clustering, and combines priority sampling and model fine-tuning to effectively mitigate the impact of unknown malicious families on the detection model in Android malware detection, providing a reliable technical solution for practical security applications.

Claims

1. An Android malware detection method based on a multi-task learning mechanism, characterized in that, Includes the following steps: Step 1: Extract the correlation between malicious families. A deep learning model based on convolutional neural network (CNN) and bidirectional long short-term memory network (Bi-LSTM) is used to extract the correlation between different malicious families through a multi-task learning mechanism. Step 2: Generate a malicious space by characterizing high-risk sample regions through feature space generation, threat region identification, and region priority ranking. Step 3, high-risk sample identification, sample screening based on regional priority and location attributes; Step 4, Decision Optimization: By combining high-risk samples and known sample anchors, the detection model is fine-tuned to improve the detection capability against unknown malicious families.

2. The method according to claim 1, characterized in that, The specific implementation of step 1 includes the following steps: Step 1.1: Parse the APK file, obtain Dalvik-style code, use regular expressions to match the invoke instruction in the code, and further determine the system function based on the package information of the target function; Step 1.2: Parse the manifest configuration file of the APK file to extract the permission information requested by the APK file; Step 1.3: For the four major Android components, by performing class hierarchy analysis on the APK file and determining the implementation classes of Activity, Service, BroadcastReceiver and ContentProvider included in the system based on the inheritance relationship; Step 1.4: Feature fusion and vectorization, combining the extracted permissions, API call sequences, component information, etc. into a unified feature representation; Step 1.5: Convert the single sample label of each sample into multiple binary classification labels required for multi-task learning; Step 1.6: Train the model hyperparameters based on a multi-task learning mechanism by constructing a hybrid embedding module containing CNN and Bi-LSTM.

3. The method according to claim 1, characterized in that, The specific implementation of step 2 includes the following steps: Step 2.1: The original high-dimensional features are converted into low-dimensional features through the trained embedding layer to construct the malicious behavior space; Step 2.2: Determine the location of high-risk areas based on the geometric center and radius of known malware family clusters; Step 2.3: Prioritize high-risk areas by calculating the deviation and density scores of the regions.

4. The method according to claim 1, characterized in that, The specific implementation of step 3 includes the following steps: Step 3.1: Use the Gaussian kernel function to sort the samples according to their distance from the center of the risk region; Step 3.2: Determine the number of samples for each region by truncating the normal distribution function, and then conduct multiple rounds of random sampling based on sample priority to ensure coverage of high-risk areas.

5. A method according to claim 1, characterized in that, The specific implementation of step 4 includes the following steps: Step 4.1: By integrating multi-task branches into a unified classification module, a deep learning model containing residual blocks is constructed. Step 4.2 involves manually analyzing high-risk samples to determine their malice. Then, by combining randomly selected known sample anchor points, the detection model is fine-tuned to alleviate the problem of knowledge forgetting while improving the detection capability against unknown malicious families.

Images