Regitive Android malicious software detection method based on comparative learning

Abstract

The invention discloses an adversarial Android malicious software detection method based on comparative learning, and the method comprises the steps: firstly, determining a perturbable feature set and a non-perturbable feature set in a feature space through defining an Android application program and all feasible perturbation sets; and then determining the architecture of a feature projection model through the obtained perturbable feature set and the non-perturbable feature set. A feature projection model is obtained based on a loss function of comparative learning and adversarial malicious software training and is used for projecting feature vectors, so that the distance between the feature vectors obtained after projection can be expressed as a feature matching degree; according to the method, the deep learning technology is utilized, high-efficiency and low-loss detection of the resistant Android malicious software can be realized, and the robustness of the Android malicious software detection method based on machine learning is improved to a great extent.

Description

Technical Field

[0001] The present invention relates to the field of Android malware defense, and in particular to an adversarial Android malware detection method based on contrastive learning. Background Art

[0002] In recent years, with the widespread use of the Android operating system, Android malware has become a serious threat to privacy and data integrity faced by users in the Android ecosystem. In order to defend against Android malware, Android malware detection algorithms based on machine learning have been proposed by researchers and have been widely used. However, these Android malware detection algorithms based on machine learning are not robust, that is, these algorithms are easily affected by adversarial Android malware generation algorithms. Adversarial Android malware generation algorithms modify detectable Android malware to bypass the detection of Android malware detection systems based on machine learning. Adversarial Android malware has brought new and severe challenges to Android malware detection systems. Android malware defense methods based on contrastive learning are one of the key technologies to alleviate this challenge.

[0003] Adversarial Android malware defense methods defend against adversarial Android malware by enhancing the robustness of machine learning-based Android malware detectors. This requires that the defense algorithm can effectively identify adversarial Android malware and that the defense algorithm does not affect its detection success rate for the original malware. Existing adversarial Android malware defense methods mainly use adversarial training and other methods, focusing on enhancing the robustness of the Android malware detection model itself. However, these methods have the following main shortcomings: First, since the Android malware detection model itself is not very complex, enhancing its detection ability for adversarial Android malware often damages its detection ability for normal software; second, the design ideas of existing methods are mainly limited to the perspective of compensating for model vulnerabilities. Since the model itself has a wide range of vulnerabilities, this idea is difficult to defend against adaptive attacks; third, existing adversarial Android malware defense methods have problems such as high computational overhead and poor generalization ability. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention adopts an adversarial Android malware detection method based on contrastive learning.

[0005] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0006] A contrastive learning-based adversarial Android malware detection method, comprising:

[0007] 1) construct two feature projection models for respectively projecting the perturbed features and the non-perturbed features of Android software into the same low-dimensional space, construct a calibration dataset including Android malware and Android benign software, train the two feature projection models based on the calibration dataset, and determine a rejection threshold during the training process;

[0008] 2) For the Android software to be detected, a preliminary detection is performed using an Android malware classifier based on machine learning. If the preliminary detection result is benign software, step 3) is continued. If the preliminary detection result is malware, no subsequent operation is performed and the final detection result is malware;

[0009] 3) extracting the perturbed features and the non-perturbed features of the Android software to be detected according to the classification results of step 1), and projecting the perturbed features and the non-perturbed features of the Android software to be detected into the same low-dimensional space using two trained feature projection models;

[0010] 4) For the perturbed features and non-perturbed features of the Android software to be detected in the low-dimensional space, a feature matching score is calculated; if the feature matching score is higher than the rejection threshold, the final detection result is malware, otherwise the final detection result is benign software.

[0011] Furthermore, in step 1), the perturbed features of the Android software are sensitive features that will change when disturbed, and the non-perturbed features are sensitive features that cannot change when disturbed; the sensitive features are permission features and encryption function features in the Android software.

[0012] Furthermore, the perturbed features and non-perturbed features of the Android software are classified using the following method: a feasible perturbation set is constructed, the Android software is iteratively perturbed from the feasible perturbation set and changes in its sensitive features are observed, the sensitive features that have changed are classified as perturbed features, and the sensitive features that cannot change are classified as non-perturbed features.

[0013] Furthermore, in the calibration data set including Android malware and Android benign software in step 2), the Android malware includes adversarial Android malware and conventional Android malware.

[0014] Furthermore, the step 2) of training two feature projection models according to the calibration data set specifically includes:

[0015] The calibration data set is used as a training data set, wherein Android malware is used as a malicious sample, Android benign software is used as a benign sample, and adversarial Android malware is used as an adversarial malware sample; according to the classification result of step 1), the perturbed features and non-perturbed features of the samples in the calibration data set are extracted to form a perturbed feature space and a non-perturbed feature space, and the feature matching score is calculated according to the perturbed feature space and the non-perturbed feature space, and the contrastive learning loss function is further calculated, and the two feature projection models are iteratively optimized based on the contrastive learning loss function.

[0016] Further, in step 2), determining the rejection threshold during the training process specifically includes:

[0017] The initial value of the rejection threshold is pre-set before training;

[0018] During the iterative optimization of the model, for each training round, the feature matching score of the current training sample is compared with the rejection threshold. If the feature matching score is higher than the rejection threshold and the current training sample is a malicious sample, the current training sample is marked as a true negative sample. If the feature matching score is higher than the rejection threshold but the current training sample is a benign sample, the current training sample is marked as a false negative sample.

[0019] In each training round, the K quantile of the feature matching scores of all true negative samples is used as the update value to update the rejection threshold and calculate the false negative impact rate;

[0020] After the training is completed, the training round with the highest false negative impact rate is selected, and the preset threshold updated in this round is used as the final preset threshold.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] (1) The present invention uses a plug-in design. Based on the conventional Android malware classifier, it focuses on improving its detection performance in terms of adversarial Android malware. This design can effectively separate the detection capabilities of adversarial Android malware from those of general malware, without compromising the detection performance of the original classifier itself in terms of general malware. This greatly improves the robustness of the Android malware detection method based on machine learning.

[0023] (2) The present invention uses contrastive learning technology to extract the unique features of adversarial Android malware, thereby achieving efficient adversarial Android malware detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The schematic diagram of the architecture of the adversarial Android malware defense method based on contrastive learning;

[0025] Figure 2 Flowchart of the adversarial Android malware defense method based on contrastive learning. DETAILED DESCRIPTION

[0026] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be pointed out that the embodiments described below are intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

[0027] In one embodiment provided by the present invention, the adversarial Android malware detection method based on contrastive learning uses a plug-in design, based on a general Android malware classifier based on machine learning, to further achieve more efficient adversarial Android malware detection. The Android malware classifier based on machine learning has been widely used in major Android systems in the field, but it is difficult to defend against adversarial Android malware. The adversarial Android malware is generated by an adversarial Android malware generation algorithm, which can modify and mutate malware that was previously detected and still maintain its original malicious functions.

[0028] like Figure 1 and Figure 2 As shown, the adversarial Android malware detection method based on contrastive learning includes the following steps:

[0029] 1) Quantify all perturbed features in the feature space by defining Android applications and feasible perturbation sets, forming perturbed feature space and non-perturbed feature space;

[0030] 2) constructing a feature projection model, projecting the perturbed feature space and the non-perturbed feature space obtained in step 1) into a low-dimensional space to compare their matching scores, calculating the loss function, and training two feature projection models based on the calibration data set;

[0031] 3) During the feature projection model training process, the rejection threshold of the matching score is determined by controlling its true negative influence ratio;

[0032] 4) For a piece of software to be detected, if it is identified as benign software by the machine learning-based Android malware classifier, this plug-in is run for further identification. If it has been classified as malware, no subsequent processing is performed and the final detection result is malware;

[0033] 5) Extract the perturbed features and the non-perturbed features of the Android software to be detected, and use the two trained feature projection models to project them into the same low-dimensional space, and calculate the feature matching score; if the feature matching score is higher than the rejection threshold, the final detection result is malware, otherwise the final detection result is benign software.

[0034] For step 1), the purpose of this step is to analyze the common sensitive features of Android software and divide the sensitive features of Android software into perturbed features and non-perturbed features. For this purpose, the present invention generates an empty Android software called a defined Android application. The defined Android application can be directly generated by the built-in Android software template of the Visual Studio development platform, and its source program file is empty. Except for the files generated by the template itself, no files for implementing other functions are attached.

[0035] Based on the defined Android applications generated by the general software template, the features of Android software are divided into perturbed features and non-perturbed features through feature extraction and perturbation analysis. The specific steps are as follows:

[0036] (1) The feature space quantization algorithm first obtains the permission information and function call information that define the Android application, and extracts its permission features and encryption function features; the permission features are whether specific permission information is used, such as camera permissions, GPS permissions, read and write external storage permissions, etc.; the encryption function features are whether specific function information is used, such as AES encryption function, MD5 encryption function, etc.

[0037] (2) The feature space quantization algorithm iteratively perturbs the Android application from the feasible perturbation set and observes the changes in features, and classifies the changed features as perturbable features;

[0038] (3) The remaining features that cannot be changed are classified as non-disturbable features.

[0039] For step 2), this step mainly focuses on determining the feature matching scores of the perturbed feature space and the non-perturbed feature space after obtaining them. However, in the original feature space, the perturbed features and the non-perturbed features are unlikely to be aligned in dimension, which makes the calculation of the feature matching score very difficult. To address these challenges, this module adopts an encoder model based on contrastive learning, which aims to project the perturbed feature space and the non-perturbed feature space into a common low-dimensional space, and then calculate the feature matching score based on the distance between the projected features. It includes the following steps:

[0040] (3-1) Construct perturbed feature space and non-perturbed feature space projection models. The perturbed feature space and non-perturbed feature space projection models further learn the mapping relationship and project the features in the feature space into a unified mapping space for calculating the matching score of the features. The perturbed feature space projection model and the non-perturbed feature space projection model use an autoencoder structure to map high-dimensional feature vectors to low-dimensional projection space;

[0041] (3-2) Training of perturbed feature space and non-perturbed feature space projection models. Based on the training dataset of the malware detector and the adversarial malware generation algorithm, a calibration dataset including Android malware and Android benign software is constructed;

[0042] (3-3) According to the perturbed feature space and the non-perturbed feature space defined in step 1), the perturbed features and the non-perturbed features of each training sample are extracted as input, and their corresponding projection vectors are projected.

[0043] (3-4) The projection vector in the projection space obtained through the above steps is used to obtain its feature matching score by calculating the corresponding distance.

[0044] The encoder model based on contrastive learning in this step is a core innovation of the present invention, which mainly includes the following two aspects:

[0045] (1) Model design. The encoder model based on contrastive learning aims to project perturbed features and non-perturbed features from a high-dimensional space to a common low-dimensional space. This module adopts two separate encoder models for the perturbed feature space and the non-perturbed feature space respectively. The architecture of each encoder model is a multi-layer perceptron model with a ReLU activation function and a dropout layer. The input layer dimension corresponds to the respective dimensions of the perturbed and non-perturbed feature spaces, and the output dimension is a very small value, specifically 32.

[0046] (2) Model training. During the training process of the encoder model, several Android malware and Android benign software are used as training samples to construct a calibration data set. In order to achieve the goal of not conflicting the adversarial malware detection capability with the normal malware detection capability, this module adopts a customized contrastive learning loss consisting of three parts, each for the corresponding data. The first part aims to minimize any negative impact on the original detection performance of the Android malware detection system. It ensures that truly benign samples are correctly identified to control the increase in false positive rate to an acceptable level. The second part aims to enhance the detection capability of the system by ensuring the correct identification of malicious samples that were previously misclassified as benign. The third part focuses on effectively detecting adversarial Android malware samples by identifying incompatibility features exhibited in their unique feature matching scores.

[0047] (i) In order to retain the benign samples (x b ), the first part of the designed contrastive learning loss will minimize the incompatibility scores of these samples. The objective formula is as follows:

[0048] D(x b )=l2(EPS(PS(x b)),EIPS(IPS(x b )))

[0049]

[0050] Where PS and IPS represent the perturbed feature space and the non-perturbed feature space, respectively. EPS and EIPS represent the encoder models used to project the perturbed feature space and the non-perturbed feature space, respectively. l2 represents the distance between the feature vectors. D(x b ) represents the feature matching score of benign samples. represents the set of benign samples in the training dataset, and |.| represents the cardinality of the set. Therefore, the first part of the customized contrastive learning loss focuses on minimizing the compatibility scores of all benign samples in the training dataset of the machine learning-based Android malware detection system.

[0051] (ii) To improve the original performance of the Android malware detection system, the second part of the designed contrastive learning loss combines the malicious samples (x m ). In this case, the model aims to identify these samples as malicious. Therefore, this part of the loss function ensures that the feature matching score of benign samples is less than that of malicious samples. This goal can be stated as:

[0052] D(x m )=l2(EPS(PS(x m )),EIPS(IPS(x m )))

[0053]

[0054] in, is the set of benign and malicious sample pairs in the training data set, where the benign and malicious sample pairs consist of any benign sample and any malicious sample in the training data set. m ) represents the feature matching score of the malicious sample. m is a hyperparameter representing a fixed distance.

[0055] (iii) In order to make the model capable of detecting adversarial Android malware, pseudo-adversarial Android malware samples are introduced into the designed contrastive learning loss function, that is, in samples that can be successfully classified as malware by the machine learning-based Android malware classifier, their feature vectors are randomly perturbed, so that the perturbed feature vectors are misclassified as normal samples by the machine learning-based Android malware classifier. After obtaining the pseudo-adversarial malware samples, the model incorporates them into the training process of the encoder model to approximate the adversarial malware samples. The goal of this training is to ensure that the adversarial malware samples (xpam ) has a higher feature matching score than regular malicious samples because adversarial malware samples show the most severe feature mismatch in features. Therefore, the third part of the contrastive learning loss is formulated as follows:

[0056] D(x pam )=l2(EPS(PS(x pam )),EIPS(IPS(x pam )))

[0057]

[0058] Among them, D(x pam ) represents the feature matching score of the adversarial malware sample, is a set of adversarial malware samples and malicious sample pairs, represents the cardinality of the set, and m is a hyperparameter.

[0059] In summary, the final training loss function of the model can be expressed as:

[0060] L=λ1L1+λ2L2+λ3L3

[0061] Among them, λ1, λ2, and λ3 are hyperparameters.

[0062] For step 3), this step mainly aims at determining the threshold of the feature matching score after constructing the encoder model of feature projection. If the feature matching score of the input sample exceeds this threshold, the model will identify it as a malicious sample. In order to determine the threshold, this module determines the threshold of the feature matching score during the model training process of step 2). It includes the following steps:

[0063] (1) Pre-set the initial value of the rejection threshold before training;

[0064] (2) During the iterative optimization process of the model, for each training round, the adversarial Android malware detection system with the plug-in is used to identify true negative samples and false negative samples from the calibration data set; specifically, for each training round, the feature matching score of the current training sample is compared with the rejection threshold. If the feature matching score is higher than the rejection threshold and the current training sample is a malicious sample, the current training sample is marked as a true negative sample. If the feature matching score is higher than the rejection threshold but the current training sample is a benign sample, the current training sample is marked as a false negative sample.

[0065] (3) For each training round, the feature matching scores of these true negative and false negative samples are counted. The threshold is calibrated according to the K quantile of the feature matching scores of the true negative samples to control the true negative impact rate. In this embodiment, K is 5%. Using this threshold, the false negative impact rate is calculated and the training round with the highest false negative impact rate is selected. The preset threshold updated in this round is used as the final preset threshold. The false negative impact rate is the proportion of false negative samples in all negative samples.

[0066] Using the above method, this embodiment tests a data set of 1,000 adversarial malware samples, and the test results are statistically shown in the following table.

[0067] Table 1: Detection success rate of adversarial malware in this embodiment

[0068]

[0069] It can be seen from the above that this embodiment has a very high detection success rate for models divided by different data sets and different feature spaces, and its detection success rate can exceed 90%.

[0070] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting adversarial Android malware based on contrastive learning, characterized in that: include: 1) construct two feature projection models for respectively projecting the perturbed features and the non-perturbed features of Android software into the same low-dimensional space, construct a calibration dataset including Android malware and Android benign software, train the two feature projection models based on the calibration dataset, and determine a rejection threshold during the training process; 2) For the Android software to be detected, a preliminary detection is performed using an Android malware classifier based on machine learning. If the preliminary detection result is benign software, step 3) is continued. If the preliminary detection result is malware, no subsequent operation is performed and the final detection result is malware; 3) extracting the perturbed features and the non-perturbed features of the Android software to be detected, and using the two trained feature projection models to project the perturbed features and the non-perturbed features of the Android software to be detected into the same low-dimensional space; 4) Calculate the feature matching score for the perturbed features and non-perturbed features of the Android software to be detected in the low-dimensional space; If the feature matching score is higher than the rejection threshold, the final detection result is malware, otherwise the final detection result is benign software.

2. The adversarial Android malware detection method based on contrastive learning according to claim 1, characterized in that: The perturbed features of the Android software are sensitive features that will change when disturbed, and the non-perturbed features are sensitive features that cannot change when disturbed; the sensitive features are permission features and encryption function features in the Android software.

3. The adversarial Android malware detection method based on contrastive learning according to claim 1, characterized in that: The perturbed features and non-perturbed features of the Android software are classified using the following method: A feasible perturbation set is constructed. The Android software is iteratively perturbed from the feasible perturbation set and the changes in its sensitive features are observed. The sensitive features that change are classified as perturbed features, and the sensitive features that cannot change are classified as non-perturbed features.

4. The adversarial Android malware detection method based on contrastive learning according to claim 1, characterized in that: In step 2), both of the feature projection models are multi-layer perceptron models with ReLU activation function and dropout layer.

5. The adversarial Android malware detection method based on contrastive learning according to claim 1, characterized in that: In step 2), the calibration data set includes Android malware and Android benign software, and the Android malware includes adversarial Android malware and conventional Android malware.

6. The adversarial Android malware detection method based on contrastive learning according to claim 4, characterized in that: Step 2) training two feature projection models according to the calibration data set specifically includes: The calibration dataset is used as a training dataset, where Android malware is used as a malicious sample x m , Android benign software is used as benign sample x b , adversarial Android malware as adversarial malware sample x pam ; According to the classification results of step 1), the perturbed features and non-perturbed features of the samples in the calibration data set are extracted to form a perturbed feature space and a non-perturbed feature space. The feature matching score is calculated according to the perturbed feature space and the non-perturbed feature space, and the contrastive learning loss function is further calculated. The two feature projection models are iteratively optimized based on the contrastive learning loss function.

7. The adversarial Android malware detection method based on contrastive learning according to claim 6, characterized in that: The contrastive learning loss function is: L=λ1L1+λ2L2+λ3L3 Among them, λ1, λ2 and λ3 are all hyperparameters, L is the contrastive learning loss, L1, L2 and L3 are the first, second and third parts of the contrastive learning loss; The first part L1 of the contrastive learning loss is specifically: D(x b )=l2(EPS(PS(x b )),EIPS(IPS(x b ))) Among them, PS and IPS represent the perturbed feature space and the non-perturbed feature space, EPS and EIPS represent the feature projection models for projecting perturbed features and non-perturbed features, respectively, l2 represents the L2 distance, D(x b ) represents the feature matching score of benign samples, represents the set of all benign samples in the training data set, and |.| represents the cardinality of the set; The first part L2 of the contrastive learning loss is specifically: D(x m )=l2(EPS(PS(x m )),EIPS(IPS(x m ))) in, represents the set of sample pairs consisting of any benign sample and any malicious sample in the training data set, D(x m ) represents the feature matching score of malicious samples, and m is a hyperparameter; The first part L3 of the contrastive learning loss is specifically: D(x pam )=l2(EPS(PS(x pam )),EIPS(IPS(x pam ))) Among them, D(x pam ) represents the feature matching score of the adversarial malware sample, Represents the set of sample pairs consisting of any adversarial malware samples and any malicious samples in the training dataset.

8. The method for detecting adversarial Android malware based on contrastive learning according to claim 6, characterized in that: In step 2), determining the rejection threshold during the training process specifically includes: The initial value of the rejection threshold is pre-set before training; During the iterative optimization of the model, for each training round, the feature matching score of the current training sample is compared with the rejection threshold. If the feature matching score is higher than the rejection threshold and the current training sample is a malicious sample, the current training sample is marked as a true negative sample. If the feature matching score is higher than the rejection threshold but the current training sample is a benign sample, the current training sample is marked as a false negative sample. In each training round, the K quantile of the feature matching scores of all true negative samples is used as the update value to update the rejection threshold and calculate the false negative impact rate; After the training is completed, the training round with the highest false negative impact rate is selected, and the preset threshold updated in this round is used as the final preset threshold.

9. The method for detecting adversarial Android malware based on contrastive learning according to claim 8, characterized in that: The false negative impact rate is the ratio of false negative samples to all negative samples.

10. The method for detecting adversarial Android malware based on contrastive learning according to claim 1, characterized in that: In step 5), the feature matching score is the L2 distance between two sets of features in the low-dimensional space.

Images