Android malware detection method based on entropy spectrum density and adaptive shrink convolution

Abstract

The present application relates to a kind of Android malware detection method based on entropy spectrum density and adaptive shrink convolution, comprising the following steps: collecting malware dataset;Dex head feature extraction;Shannon entropy distribution feature extraction;Markov image combined gray image feature extraction, Android manifest file analysis module;Using adaptive soft voting method to carry out integrated learning, and predict final result.The present application introduces a kind of light multi-source fast Android malware detection method, uses multiple files in an Android application package, extracts relevant features contained in file from multiple dimensions, such as information in file header and power spectrum density of executable file structure entropy, so that the extraction of feature is more comprehensive.

Description

Technical Field

[0001] This invention relates to the field of Android malware detection technology, specifically an Android malware detection method based on entropy spectral density and adaptive shrinking convolution. Background Technology

[0002] Since its release in 2008, Android has become the most popular operating system for smart mobile devices. In 2021, 89.6% of Chinese internet users' smartphones used Android. Due to Android's open ecosystem, coarse-grained permission management, and ability to access third-party code, numerous security attack surfaces have emerged, seriously threatening the integrity of Android applications. Currently, most malware detection on the Android platform relies on authoritative security organizations to establish massive virus databases and utilize cloud technology to provide services to users, and only detects applications from official app stores. However, third-party applications still pose serious security threats to users. Furthermore, most current offline Android malware detection methods use complex models that are difficult to run on user terminals lacking computing resources. Therefore, Android malware detection remains a significant challenge.

[0003] Traditional antivirus technologies largely rely on signature-based and heuristic detection methods to quickly and effectively detect known malware. Signature-based detection primarily involves extracting signatures from malware and building malware databases. However, some malware can hide within the system using various obfuscation and camouflage techniques, making it undetectable. Machine learning algorithms typically have fewer than three layers of computational units, limiting their processing power for raw data. Therefore, the performance of machine learning models heavily depends on extracted features, and malware producers can bypass well-trained machine learning models by continuously updating their deception techniques, harming users and companies. Given the increasing difficulty of Android malware detection, building a robust and transparent detection model or system solely through traditional machine learning techniques is not easy. Deep learning, however, has become one of the mainstream algorithms in recent years. Unlike traditional machine learning techniques, deep learning's feature extraction methods can learn feature representations from raw input data without requiring prior knowledge. Furthermore, based on recognizing specific patterns in known malware, it has the ability to detect previously undiscovered malware types, offering better performance in terms of detection efficiency and effectiveness – a key advantage of deep learning.

[0004] Patent application CN202210483736.3 proposes an Android malware detection method based on convolutional neural networks. It employs a novel logic design, addressing the low accuracy and low recall issues present in current Android malware detection methods. However, this patent suffers from a problem: most current Android malware detection technologies are signature-based, and deep learning-based detection technologies consume excessive resources, making them unsuitable for use on user terminals. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide an Android malware detection method based on entropy spectral density and adaptive shrinking convolution, which aims to solve the problems of not being able to run on user terminals and to significantly improve detection speed and reduce resource consumption without sacrificing accuracy.

[0006] To solve the above technical problems, the present invention adopts the following technical solution:

[0007] An Android malware detection method based on entropy spectral density and adaptive shrinking convolution includes the following steps:

[0008] Step S1: Collect malware dataset;

[0009] Step S2: Extract and parse the header information of the classes.dex file of the apk file in the malware dataset, and encode the key fields in the header information to obtain the header features;

[0010] Step S3: Calculate the entropy signal using the classes.dex file of the apk file in the malware dataset as a binary stream, and extract the entropy spectral density feature of the entropy signal using the maximum entropy method.

[0011] Step S4: Read the apk file in the malware dataset in binary format to generate a grayscale image. Then, use a Markov algorithm to generate a Markov image by performing transition probabilities on the apk file in the malware dataset. Combine the grayscale image and the Markov image to obtain a Markov image combined with the grayscale image feature image.

[0012] Step S5: Decode and parse the AndroidManifest.xml file in the apk file of the malware dataset, extract permission and intent keywords, and then encode them as features using the Bag-of-words model;

[0013] Step S6: Input the head features, entropy spectral density features, Markov images, grayscale image features, and features encoded by the Bag-of-words model obtained in steps S2, S3, S4, and S5 into an adaptive shrinking convolutional unit for noise reduction and feature extraction.

[0014] Step S7: Use the features extracted in the previous step to train the classifier and obtain the adaptive soft voting model;

[0015] Step S8: Use an adaptive soft voting model to detect Android malware.

[0016] Furthermore, step S2 includes the following steps S2A-1 to S2A-3:

[0017] Step S2A-1: Verify the magic number field of the classes.dex file to determine if it is a valid classes.dex file. If it is, proceed to the next step; otherwise, skip the sample.

[0018] Step S2A-2: Parse the classes.dex file according to the DexHeader structure definition in DexFile.h, obtain all fields in the structure, and parse out the file signature, link section, mapping item, type identifier, string identifier and prototype identifier size and offset information;

[0019] Step S2A-3: The size and offset information of the parsed file signature, link section, mapping item, type identifier, string identifier and prototype identifier are encoded in hexadecimal and normalized to obtain a one-dimensional gray matrix, which is the header feature of the classes.dex file.

[0020] Furthermore, step S3 includes the following steps S3A-1 to S3A-2:

[0021] Step S3A-1: Input the binary stream of the classes.dex file as a signal into the entropy distribution feature extraction module;

[0022] Step S3A-2: Extract the Shannon entropy sequences and entropy spectral density of malicious and benign software.

[0023] Furthermore, the formula for Shannon entropy in step S3A-2 is expressed as follows:

[0024]

[0025] Where H(x) is the entropy of variable X, p(x) i ) is the possible outcome x of variable X. i The probability of occurrence, where i represents the number of outcomes (i varies between 1 and n);

[0026] The formula for calculating entropy spectral density is as follows:

[0027]

[0028] Where P(f) is the entropy spectral density, P(m) is the output power of the filter when the value is the m-th order fluctuation period, and γ m，k Let m = 0, 1, ..., M (M being the filter coefficients corresponding to the optimal filter order), P m and γ m，k The solution is obtained by solving the Yule-Walker equation using the Burg process.

[0029] Furthermore, step S4 includes the following steps S4A-1 to S4A-3:

[0030] Step S4A-1: Read the apk file in binary format, using 8-bit unsigned integers as one byte unit, fix the line width and width, and finally generate a two-dimensional array; the value range of each element in the array is 0-255, which is exactly the value range of a grayscale image, thus obtaining a grayscale image.

[0031] Step S4A-2: Read the APK file as a character code sequence, then count the byte transfer frequency of each APK file, and finally calculate the transfer probability matrix of each byte to obtain a 256*256 Markov image.

[0032] Step S4A-3: Concatenate the grayscale image and the Markov image using 16-bit binary data respectively, extract 8-bit binary values ​​from each image and map them to a pixel, then compress the image using the transition probability to generate a 256*256 Markov image combined with the grayscale image feature image.

[0033] Furthermore, step S5 includes the following S5A-1 to S5A-4:

[0034] Step S5A-1: Decrypt the AndroidManifest.xml file into a valid XML document, parse it, extract the application's permission usage rights, components, and intents, and perform statistics.

[0035] Step S5A-2: Using the idea of ​​the Bag-of-words model, the occurrence of keywords for each permission, component, and intent is treated as an independent probability;

[0036] Step S5A-3: Extract all permissions and intents from the malware dataset and filter out keywords with a frequency of less than 2 to form a dictionary containing N keywords.

[0037] Step S5A-4: Perform BOW extraction on each AndroidManifest.xml file in the malware dataset, and replace keywords not in the dictionary with UNK to form an N+1 dimensional feature vector.

[0038] Furthermore, in step S6, the adaptive shrinking convolutional unit, for each convolutional layer, uses a dynamic convolution method to non-linearly stack convolutional kernels based on attention, thereby improving the model's expressive power. Let the traditional static convolution be represented as:

[0039]

[0040] Where W and b are the weight matrix and bias vector, respectively, and g is an activation function;

[0041] By aggregating multiple (K) linear functions Dynamic convolution is defined as follows:

[0042]

[0043]

[0044] Where, π k It is the k-th linear equation Attention weights, π k (x)∈[0,1], and Weight and bias It is the input function and shares the same attention, with attention weights π. k (x) is not fixed, but varies with each input x, representing a linear model for a given input. Optimal aggregation, aggregation model It is a non-linear function;

[0045] The adaptive shrinking convolutional unit also introduces a soft-thresholding activation function. This function sets the features corresponding to the noise to zero using soft thresholding, and dynamically sets a threshold for each sample individually based on the situation of each sample through an attention mechanism. This is integrated into the unit as a non-linear transformation layer. The formula for the soft threshold is expressed as:

[0046]

[0047] Where x is the input feature, y is the output feature, and τ is the threshold.

[0048] Furthermore, the method for training the adaptive software voting model in step S7 includes the following steps: steps S7A-1 to S7A-4:

[0049] Step S7A-1: Calculate the binary cross-entropy of all samples in the malware dataset. The formula for binary cross-entropy is:

[0050]

[0051] Where N is the number of samples, pi is the probability that the i-th sample is a positive example, and yi is the label of the i-th sample;

[0052] Step S7A-2: Calculate the probability {p1,p2,…,pj} and weight W of the positive examples obtained by inputting all features of all samples into the classifier, and then use the Hadamard product of the Softmax operation to obtain the positive example probability p after soft voting. vote The calculation formula is as follows:

[0053]

[0054]

[0055] Where M is the total number of features across all samples, M = number of samples * number of features, σ(W) j Let p be the Softmax function corresponding to the j-th feature (j = 1, 2, ..., M). j Let W be the probability of a positive example corresponding to the j-th feature (j = 1, 2, ..., M), and W be the weight.

[0056] Step S7A-3: Calculate the loss of dynamic weighted soft voting. The calculation formula is as follows:

[0057]

[0058] in, The binary cross-entropy loss {l1, l2, ..., l} is derived for M features. j The average of the absolute values ​​of the differences between the values ​​after the Softmax operation and the values ​​after the Softmax operation of the weights, where μ is the weight parameter and L is the weight. BCE (p vote ,y) is the sum of the binary cross-entropy of the soft voting results;

[0059] Step S7A-4: Train the classifier by minimizing the loss function Loss, and then obtain the trained adaptive software voting model.

[0060] Compared with the prior art, the present invention, by adopting the above technical solution, has the following advantages:

[0061] (1) This invention introduces a lightweight, multi-source, fast Android malware detection method, which uses multiple files in an Android application package to extract relevant features contained in the files from multiple dimensions, such as information in the file header and the power spectral density of the structural entropy of the executable file, making the feature extraction more comprehensive.

[0062] (2) This invention studied the information in each field of the DEX file header and found that there are some key features in the DEX file header that can be used for malware detection. Therefore, a DEX header parser is proposed to extract key features from the DEX file header;

[0063] (3) A Markov image combined with grayscale image feature image is proposed. First, the APK file is converted into a grayscale image. Then, the APK file is converted into a Markov image. The grayscale image and the Markov image of the two selected feature files are concatenated with 16-bit binary data respectively. 8-bit binary values ​​are extracted and mapped to a pixel. The transition probability is then used to compress the image to generate a 256*256 Markov image combined with grayscale image feature image.

[0064] (4) This invention introduces an adaptive shrinking convolutional neural network. This network can dynamically adjust the convolutional kernel weights and activation function thresholds through the attention mechanism, so that the convolutional network has the ability to denoise and improves the expressive power of the neural network model. While slightly increasing the computational complexity of the convolutional layer, it greatly improves the ability of the convolutional network to extract entropy features, so that this method only needs to use shallow neural networks and reduces the overall time complexity of the model.

[0065] (5) A new adaptive soft voting method is proposed, which can dynamically change the weight of each basic feature during the training process, overcome the jitter caused by the large performance gap of the basic model in traditional soft voting, and significantly improve the performance of soft voting.

[0066] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0068] Figure 2 This is a schematic diagram of the Android application package structure;

[0069] Figure 3 This is a schematic diagram of the DEX file structure;

[0070] Figure 4 This is a schematic diagram of the structure of a Markov image combined with a grayscale image. Detailed Implementation

[0071] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0072] Example 1:

[0073] like Figure 1The method for fast Android malware detection based on entropy spectral density and adaptive shrinking convolution, as shown, includes the following steps:

[0074] Step S1: Collect and create a malware dataset;

[0075] This invention uses three publicly available datasets: the CICModalDroid 2020 dataset and the CIC-InvesAndMal2019 dataset, both built by the Canadian Cyber ​​Security Institute (CIC), and the Drebin dataset from the Institute for Systems Security at the Technical University of Brunswick. The CICModalDroid 2020 dataset contains over 17,341 Android samples, including VirustoTalService, Contagio Security Blog, AMD, Maldozer, and other datasets used in recent research contributions. The CIC-InvesAndMal2019 dataset contains 10,854 samples (4,354 malware and 6,500 benign software). The Drebin dataset contains 5,560 malicious samples.

[0076] Step S2, Dex head feature extraction;

[0077] An Android package is a ZIP archive with the .apk extension that contains all the content of an Android application. The assets directory stores static resource files required by the application, such as images. Resource files in the res directory are compiled into binary format, generating corresponding index IDs in the R.java file. The lib directory contains library files written in C / C++. META-INF stores the application's signature information; signature verification is performed before application installation to ensure the integrity and security of the Android application package. The resources.arsc file records the mapping between resource files and their IDs in the res directory; its structure is illustrated in the diagram below. Figure 2 As shown.

[0078] classes.dex is the executable file for the Android Runtime (ART) and Dalvik virtual machine. The compact Dalvik executable file format is designed for systems with limited memory and processor speed. Essentially, classes.dex contains all the program logic of an Android application. A given Android application is typically written in Java and compiled into bytecode by the dx tool. The Java compiler compiles all Java source files into Java bytecode (.class files), and then the dx tool converts them from Java bytecode into Dalvik-compatible bytecode .dex files. The dx tool eliminates redundant information in the classes. In the dex file, all .class files are wrapped into a single file, merging the header information of the .class files and sharing a constant pool and index. Therefore, most of the code logic of an Android application is contained in the classes.dex file, as shown in the diagram below. Figure 3 As shown.

[0079] Step S2, which distinguishes between malicious and benign applications based on the header information of the dex file, contains three branches, S2A, which include steps S2A-1 to S2A-2 to S2A-3 as follows:

[0080] Step S2A: Extract and parse the header information of the classes.dex file, and encode the key fields as header features;

[0081] Step S2A-1: The Dex magic number field is checked to determine whether it is a valid dex file. The dex magic number is usually the first 8 bytes of the dex file, represented by 8 one-byte unsigned numbers. Its value is a combination of the dex string and the file version number, such as "64 65 78 0A 30 33 35 00". After conversion using the ASCII code table, it becomes "dex\n\035\0". The version number is used to support the system in recognizing and parsing dex files of different version formats. The 035 version format was added in Android 5.0.

[0082] Step S2A-2: After verifying that the dex file is valid, parse it according to the DexHeader structure definition in DexFile.h and obtain all fields in the structure.

[0083] Step S2A-3: The size and offset information of the parsed file signature, link segment, mapping item, type identifier, string identifier, prototype identifier, etc. are encoded in hexadecimal and normalized to obtain a one-dimensional gray matrix, which is the feature extracted from the dex header.

[0084] Step S3: Extraction of Shannon entropy distribution features;

[0085] Malware of the same type often shares similar malicious code segments. The variations in these segments are ultimately reflected in the binary information of the dex file. Since malware exists within its binary data stream, these malicious code characteristics are also reflected in the entropy sequence calculated by segmenting the binary data stream. Measuring the information content of the classes.dex file's binary stream yields an entropy sequence. Because the sizes of classes.dex files vary across different Android programs, the lengths of the resulting entropy sequences also differ.

[0086] Step S3 contains three branches S3A, and step S3A includes the following steps from S3A-1 to S3A-2 to S3A-3:

[0087] Step S3A: Calculate the entropy signal of classes.dex as a binary stream, and extract the entropy spectral density of the entropy signal as a feature using the maximum entropy method. Then, perform noise reduction and feature extraction on this feature using adaptive shrinking convolution to obtain the entropy feature.

[0088] Step S3A-1 involves inputting the binary stream of the dex file as a signal into the entropy distribution feature extraction module;

[0089] Step S3A-2: Extract the Shannon entropy sequences and entropy spectral density of malicious and benign software;

[0090] In step S3A-3, the obtained entropy spectral density is denoised and feature extracted through adaptive shrinking convolution, and then used as one of the input vectors of the backend classifier.

[0091] Entropy is a measure of the randomness or uncertainty of a variable. Shannon linked information and uncertainty; if we can measure the uncertainty of something, we can also measure the information content of a certain piece of information. The formula for Shannon entropy is expressed as:

[0092]

[0093] Where H(x) is the entropy of variable X, p(x) i ) is the possible outcome x of variable X. i The probability of occurrence, where i represents the number of outcomes, varying between 1 and n.

[0094] The power spectral density (PSD) of a signal describes the power present in the signal as a function of frequency. Methods for calculating the PSD mainly include the Fast Fourier Transform (FFT), Welch method, multi-window method, and maximum entropy method. The first three belong to the periodogram method. Since the periodogram method is a method for estimating the autocorrelation of a signal over a finite length, it requires truncating or windowing the signal sequence so that the estimated power spectrum is the convolution of the true spectrum and the windowed spectrum. Therefore, its ability to produce accurate power spectrum estimates is limited. This paper uses the maximum entropy method to calculate the PSD P(f), and the calculation formula is expressed as:

[0095]

[0096] Where P(m) is the output power of the filter when the value is the m-th oscillation period, γ m，k Let m = 0, 1, ..., M, where M is the corresponding filter coefficient when the filter order is optimal.

[0097] Step S4: Markov image combined with grayscale image feature extraction;

[0098] Step S4A, Markov image combined with grayscale image feature extraction: First, the apk file is read in binary format to generate a grayscale image; then, the transition probability is calculated using Markov to generate a Markov image combined with grayscale image features.

[0099] Step S4A-1, grayscale image feature extraction; First, using the given binary code file DEX, with 8-bit unsigned integers as one byte unit, and fixing the row and width, a two-dimensional array is generated; the value range of each element in this array is 0-255, which is exactly the value range of a grayscale image;

[0100] Step S4A-2, Markov image extraction: First, the APK file is read in as a character code sequence, then the byte transition frequency of each APK file is counted, and finally, a 256*256 Markov image is obtained by calculating the transition probability matrix of each byte.

[0101] Step S4A-3: Markov image combined with grayscale image extraction; First, convert the APK file into a grayscale image; then convert the APK file into a Markov image; concatenate the selected grayscale image and Markov image using 16-bit binary representations respectively, extract 8-bit binary values ​​from each and map them to a pixel, then compress using transition probabilities to generate a 256*256 Markov image combined with a grayscale image feature image, such as... Figure 4 As shown.

[0102] Step S5, Android manifest file parsing module;

[0103] AndroidManifest.xml is the manifest file for Android applications. It describes the various components of the application and defines their appearance, such as component name, theme, launch type, component actions, and launch intent. This manifest file also declares the application's attributes and permissions.

[0104] Step S5 contains three branches S4A, and step S4A includes the following steps from S4A-1 to S4A-2 to S4A-3 to S4A-4:

[0105] Step S5A: Decode and parse the AndroidManifest.xml file, extract permission and intent keywords, and encode them as features using the Bag-of-words model;

[0106] Step S5A-1: AndroidManifest.xml is usually encrypted. This file is decrypted into a valid XML document and parsed to extract the application's permissions, components, and intents, and then statistically analyzed.

[0107] Step S5A-2: Since the set of permission and intent keywords is small, a sparse representation is adopted to reduce the model complexity. The idea of ​​the Bag-of-words model is used to treat the occurrence of each permission and intent keyword as an independent probability.

[0108] Step S5A-3: Extract all permissions and intents from the dataset and filter out keywords with a frequency of less than 2 to form a dictionary containing N keywords.

[0109] Step S5A-4: BOW extraction is performed on each application manifest file. Keywords not in the dictionary are replaced with UNK to form an N+1 dimensional feature vector. Adaptive shrinking convolutional units are used for deep extraction and classification.

[0110] Step S6: Use an adaptive soft voting method to perform ensemble learning and predict the final result.

[0111] For a complex file like classes.dex, its entropy signal reflects the overall information of the file, and much noise unrelated to malicious code fragments is mixed in the entropy representation of the malicious code fragments to be extracted. To address this problem, the features collected in the previous step need to be denoised to improve feature learning ability. The classic wavelet thresholding denoising method mainly consists of three steps: wavelet decomposition, soft thresholding, and wavelet reconstruction. In this denoising method, constructing a suitable filtering operator and setting an appropriate soft threshold is a challenging problem. To solve this problem, this invention proposes an adaptive shrinking convolutional unit.

[0112] Soft voting, also known as weighted average probability voting, is a voting method that uses output class probabilities for classification. It obtains a weighted average of the probabilities of each class by inputting weights and selects the class with the larger value. Soft voting returns the class label as Argmax, which is the sum of the predicted probabilities, and is achieved by using the output class probabilities.

[0113] Traditional static weighted average probability voting uses manually determined weights and typically employs multiple base models of the same type. However, this paper uses different models to extract different types of features. Since different types of features and models have different training curves and fitting capabilities, the voting weights need to be dynamically adjusted during training. To address this, this invention proposes a dynamic weighted soft voting ensemble method, where voting weights are automatically determined by the gradient descent principle, and a new loss function is defined for them.

[0114] Step S6 contains three branches S6A, and step S6A includes the following steps from S6A-1 to S6A-2 to S6A-3 to S6A-4:

[0115] Step S6A: Perform ensemble learning through dynamic weighted soft voting;

[0116] Step S6A-1: An adaptive shrinking convolutional unit is used for noise reduction to improve feature learning ability;

[0117] Step S6A-2 introduces a soft-threshold activation function, which sets the features corresponding to these noises to zero by means of soft thresholding;

[0118] To denoise the features, this paper introduces a soft-threshold activation function based on the research of deep residual shrinkage networks. Soft thresholding sets the features corresponding to noise to zero, and an attention mechanism dynamically sets a threshold for each sample individually, integrating it as a non-linear transformation layer into this unit. Let x be the input feature, y be the output feature, and τ be the threshold. The formula for soft thresholding is expressed as:

[0119]

[0120] Where x is the input feature, y is the output feature, and τ is the threshold.

[0121] Step S6A-3: The threshold is dynamically set for each sample individually based on the situation of each sample through the attention mechanism, and integrated into this unit as a nonlinear transformation layer;

[0122] Step S6A-4: Obtain the value of the binary cross-entropy loss through the model and perform Softmax operation;

[0123] Step S6A-5: Calculate the average of the absolute values ​​of the differences after the Softmax operation;

[0124] Step S6A-6, and multiply by the weight parameter μ to obtain the value;

[0125] Step S6A-7: Calculate the sum of the binary cross-entropy of this value and the soft voting result to obtain the loss of the dynamically weighted soft voting;

[0126] Steps S6A-8 involve mixing the method with an adaptive shrinking convolutional unit to obtain an adaptive soft voting method.

[0127] Step S6A-9: Predict the final result;

[0128] Traditional static weighted average probability voting uses manually determined weights and typically employs multiple base models of the same type. However, this paper proposes a dynamic weighted soft voting ensemble method. This method extracts different types of features using different models, and since different features and models have different training curves and fitting capabilities, the voting weights need to be dynamically adjusted during training. To address this, this paper proposes a dynamic weighted soft voting ensemble method where voting weights are automatically determined by the gradient descent principle, and a new loss function is defined for them.

[0129] Let N be the sample size, p i Let y be the probability that the i-th sample is a positive example. i Let be the label of the i-th sample.

[0130]

[0131] Calculate the probability {p1, p2, ..., p} of the positive examples obtained from M models. j The positive probability p after soft voting is obtained by performing the Hadamard product of the weights W and Softmax. vote , Softmax function σ(W).

[0132]

[0133]

[0134] The binary cross-entropy loss {l1, l2, ..., l} derived from M models j The absolute value of the difference between the value obtained by the Softmax operation and the value obtained by the weights after the Softmax operation is averaged and multiplied by the weight parameter μ. Finally, the sum of the binary cross-entropy of this value and the soft voting result is calculated.

[0135]

[0136] To conduct experiments more effectively and to verify the classification results accordingly, two metrics are mainly used: F1 score and accuracy. Here, N is the total number of samples; the specific calculation of the F1 score is shown in Equation (1), the specific accuracy in Equation (2), the specific precision in Equation (3), and the specific recall in Equation (4).

[0137]

[0138]

[0139]

[0140]

[0141] In the above formula, TP is a true example, FP is a false counterexample, FN is a false positive example, and TN is a true counterexample.

[0142] All experimental parameters in this patent are set as follows: learning rate is set to 0.1, number of iterations is 100, weight decay coefficient is 2e-7, batch size is set to 16. In order to ensure the consistency of the experiments, all experiments are conducted in the same experimental environment, and the ratio of training set to test set is 6:4.

[0143] Method ACC F1-Score CNN+LeNet 92.54％ 95.28％ CNN 93.23％ 95.24％ LSTM 95.12％ 98.45％ Our Method 96.64％ 98.93％

[0144] By comparing with the methods mentioned above, and considering both ACC and F1 scores, it is evident that this method achieves a better balance between accuracy and speed compared to other methods. Furthermore, this method uses only shallow convolutional networks, meeting the requirements for running on users' Android devices.

[0145] The above description provides examples of the preferred embodiments of the present invention. Parts not detailed herein are common knowledge to those skilled in the art. The scope of protection of the present invention is determined by the claims. Any equivalent modifications based on the technical teachings of the present invention are also within the scope of protection of the present invention.

Claims

1. An Android malware detection method based on entropy spectral density and adaptive shrinking convolution, characterized in that, Includes the following steps: Step S1: Collect malware dataset; Step S2: Extract and parse the header information of the classes.dex file of the apk file in the malware dataset, and encode the key fields in the header information to obtain the header features; Step S3: Calculate the entropy signal using the classes.dex file of the apk file in the malware dataset as a binary stream, and extract the entropy spectral density feature of the entropy signal using the maximum entropy method. Step S4: Read the apk file in the malware dataset in binary format to generate a grayscale image. Then, use a Markov algorithm to generate a Markov image by performing transition probabilities on the apk file in the malware dataset. Combine the grayscale image and the Markov image to obtain a Markov image combined with the grayscale image feature image. Step S5: Decode and parse the AndroidManifest.xml file in the apk file of the malware dataset, extract permission and intent keywords, and then encode them as features using the Bag-of-words model; Step S6: Input the head features, entropy spectral density features, Markov images, grayscale image features, and features encoded by the Bag-of-words model obtained in steps S2, S3, S4, and S5 into an adaptive shrinking convolutional unit for noise reduction and feature extraction. In step S6, the adaptive shrinking convolutional unit, for each convolutional layer, uses a dynamic convolution method to non-linearly stack convolutional kernels based on attention, enhancing the model's expressive power. Let the traditional static convolution be represented as: ； in, and These are the weight matrix and the bias vector. It is an activation function; By aggregating multiple linear functions Dynamic convolution is defined as follows: ； ； in, It is the first Linear equations Attention weights ,and Weight and bias It is the input function and shares the same attention, attention weights. It's not fixed, but rather specific to each input. The changes represent a linear model given an input. Optimal aggregation, aggregation model It is a non-linear function; The adaptive shrinking convolutional unit also introduces a soft-thresholding activation function. This function sets the features corresponding to the noise to zero using soft thresholding, and dynamically sets a threshold for each sample individually based on the situation of each sample through an attention mechanism. This is integrated into the unit as a non-linear transformation layer. The formula for the soft threshold is expressed as: ； Where x represents the input feature and y represents the output feature. The threshold is used; Step S7: Train the classifier using the features extracted in the previous step to obtain the adaptive soft voting model; The method for training the adaptive software voting model in step S7 includes the following steps: steps S7A-1 to S7A-4: Step S7A-1: Calculate the binary cross-entropy of all samples in the malware dataset. The formula for binary cross-entropy is: ； Where N is the number of samples, p i Let y be the probability that the i-th sample is a positive example. i Let be the label of the i-th sample; Step S7A-2: Calculate the probability {p1, p2, ..., p} of the positive examples obtained by inputting all features of all samples into the classifier. j The positive probability after soft voting is obtained by combining the weights W and the Hadamard product of the Softmax operation. The calculation formula is as follows: ； = ； Where M is the total number of features across all samples, M = number of samples * number of features. Let M be the softmax function corresponding to the j-th feature, where j = 1, 2, ..., M. Let W be the probability of a positive example corresponding to the j-th feature, where j = 1, 2, ..., M, and W is the weight. Step S7A-3: Calculate the loss of dynamic weighted soft voting. The calculation formula is as follows: ( ,y)+ ； in, The binary cross-entropy loss derived from M features { The average of the absolute values ​​of the differences between the values ​​after the Softmax operation and the values ​​after the Softmax operation on the weights. For weight parameters, ( ,y) is the sum of the binary cross-entropy of the soft voting results; Step S7A-4: Train the classifier by minimizing the loss function Loss, and then obtain the trained adaptive software voting model; Step S8: Use the adaptive soft voting model to detect Android malware.

2. The Android malware detection method based on entropy spectral density and adaptive shrinking convolution according to claim 1, characterized in that, Step S2 includes the following steps S2A-1 to S2A-3: Step S2A-1: Verify the magic number field of the classes.dex file to determine if it is a valid classes.dex file. If it is, proceed to the next step; otherwise, skip the sample. Step S2A-2: Parse the classes.dex file according to the DexHeader structure definition in DexFile.h, obtain all fields in the structure, and parse out the file signature, link section, mapping item, type identifier, string identifier and prototype identifier size and offset information; Step S2A-3: The size and offset information of the parsed file signature, link section, mapping item, type identifier, string identifier and prototype identifier are encoded in hexadecimal and normalized to obtain a one-dimensional gray matrix, which is the header feature of the classes.dex file.

3. The Android malware detection method based on entropy spectral density and adaptive shrinking convolution according to claim 1, characterized in that, Step S3 includes the following steps S3A-1 to S3A-2: Step S3A-1: Input the binary stream of the classes.dex file as a signal into the entropy distribution feature extraction module; Step S3A-2: Extract the Shannon entropy sequences and entropy spectral density of malicious and benign software.

4. The Android malware detection method based on entropy spectral density and adaptive shrinking convolution according to claim 3, characterized in that, The formula for Shannon entropy in step S3A-2 is expressed as follows: ； Where H(x) is the entropy of variable X, The possible outcomes of variable X. The probability of occurrence, where i represents the number of outcomes, and i varies between 1 and n; The formula for calculating entropy spectral density is as follows: ； in, For entropy spectral density, Let the value be the output power of the filter when the m-th oscillation period is reached. Let m = 0, 1, ..., M, where M is the filter coefficient corresponding to the optimal filter order. and The solution is obtained by solving the Yule-Walker equation using the Burg process.

5. The Android malware detection method based on entropy spectral density and adaptive shrinking convolution according to claim 1, characterized in that, Step S4 includes the following steps S4A-1 to S4A-3: Step S4A-1: Read the apk file in binary format, using 8-bit unsigned integers as one byte unit, fix the line width and width, and finally generate a two-dimensional array; the value range of each element in the array is 0-255, which is exactly the value range of a grayscale image, thus obtaining a grayscale image. Step S4A-2: Read the APK file as a character code sequence, then count the byte transfer frequency of each APK file, and finally calculate the transfer probability matrix of each byte to obtain a 256*256 Markov image. Step S4A-3: Concatenate the grayscale image and the Markov image using 16-bit binary data respectively, extract 8-bit binary values ​​from each image and map them to a pixel, then compress the image using the transition probability to generate a 256*256 Markov image combined with the grayscale image feature image.

6. The Android malware detection method based on entropy spectral density and adaptive shrinking convolution according to claim 1, characterized in that, Step S5 includes the following S5A-1 to S5A-4: Step S5A-1: Decrypt the AndroidManifest.xml file into a valid XML document, parse it, extract the application's permission usage rights, components, and intents, and perform statistics. Step S5A-2: Using the idea of ​​the Bag-of-words model, the occurrence of keywords for each permission, component, and intent is treated as an independent probability; Step S5A-3: Extract all permissions and intents from the malware dataset and filter out keywords with a frequency of less than 2 to form a dictionary containing N keywords. Step S5A-4: Perform BOW extraction on each AndroidManifest.xml file in the malware dataset, and replace keywords not in the dictionary with UNK to form an N+1 dimensional feature vector.

Images