A malware detection method based on malicious behavior enhanced pre-training model

Abstract

The application discloses a malware detection method based on malicious behavior enhanced pre-training model, which comprises the following steps: firstly, extracting API call sequences from an API call sequence data set; secondly, forming a corpus by all API call sequences, and generating an API tuple set reflecting software malicious behaviors according to statistical rules; thirdly, pre-training a malware detection model by using a public data set, and then inputting a no-label API call sequence into the malware detection model to train a masked language model of the malware detection model; finally, fine-tuning the malware detection model by using a labeled API call sequence, and realizing binary classification through linear dimension reduction to complete the detection of the malware. The application can make the malware detection model fully learn the software malicious behavior characteristics, accurately identify the malicious behaviors and detect the malware.

Description

Technical Field

[0001] This invention relates to the field of software security protection technology, specifically to a malware detection method based on a pre-trained model. Background Technology

[0002] With the rapid development of the internet, the number and variants of malware have also experienced explosive growth. According to data from the German security technology company AV-TEST, as of July 2023, the number of malware exceeded 900 million, with malware on the Windows platform accounting for over 75%. Because the Windows operating system is widely used in various fields, malware can cause serious damage and threats to user devices, data, and resources. Despite the significant efforts made by Microsoft and software security companies to protect user operating systems, a report from the US security technology company McAfee shows that the growth rate of malware has not been curbed, mainly because malware authors have adopted more advanced anti-detection technologies, increasing the probability of malware escaping detection. Therefore, employing advanced and effective methods to detect malware is essential.

[0003] Malware inevitably possesses some form of malicious behavior characteristics. Therefore, one of the most common and effective methods is to analyze the API call sequences generated during malware runtime, extract patterns or rules that reflect its behavioral characteristics, and use these patterns or rules to identify whether unknown samples are malicious. In recent years, deep neural networks such as Convolutional Neural Networks (CNNs) and Bidirectional Long Short-Term Memory Networks (BiLSTMs) have demonstrated powerful feature extraction and rule discovery capabilities, achieving good accuracy in malware detection. Simultaneously, with self-supervised pre-trained models achieving state-of-the-art results in natural language processing tasks such as text classification, research in this field has begun to explore the use of pre-trained models for malware detection. Pre-trained models, such as the pre-trained Task Masked Language Model (MLM) used in Bidirectional Encoder Representation (BERT), enable the model to learn the contextual semantic information of API call sequences, and research shows that incremental pre-training allows models to better adapt to downstream tasks. Pre-trained models based entirely on attention mechanisms not only consider the semantic information of API call sequences but also discover the relationships between API calls. However, in malware detection, a single API call is unlikely to represent whether it exhibits a certain form of malicious behavior. Existing pre-trained models do not take this characteristic into account, leading to potentially suboptimal detection results. Therefore, how to enable pre-trained models to learn the malicious behavior characteristics of software more accurately and effectively is an urgent problem to be solved. Summary of the Invention

[0004] To overcome the shortcomings of the above technologies, this invention proposes a pre-trained model that incorporates the behavioral characteristics of malicious software, which can accurately detect malicious software in real samples.

[0005] This invention is achieved through the following technical solution:

[0006] This invention relates to a malware detection method based on a malicious behavior-enhanced pre-trained model, comprising the following steps:

[0007] Step 1: Extract API call sequences from the API call sequence dataset.

[0008] Step 2: Construct a corpus from all API call sequences extracted from the API call sequence dataset, and generate a set of API tuples that can reflect malicious software behavior according to statistical rules.

[0009] Step 3: Pre-train the malware detection model using a public dataset, then continue pre-training using unlabeled API call sequences, and finally train the malware detection model using a masked language model (MLM) by inputting the unlabeled API call sequences into the malware detection model.

[0010] Step 4: Fine-tune the malware detection model using a labeled API call sequence, and perform binary classification through linear dimensionality reduction to complete malware detection.

[0011] The API call sequence dataset refers to a collection of API call sequences extracted from real sample software running in a sandbox. It includes an unlabeled API call sequence set for pre-training of malware detection models and a labeled API call sequence set for fine-tuning.

[0012] The statistical rules mentioned above generate a set of API tuples that can reflect malicious software behavior. This invention proposes TF-TW (Term Frequency-Term Weight) to evaluate which API tuples are helpful in detecting malicious behavior. Specifically, firstly, all API call sequences in the API call sequence dataset are divided into tuples, for example, sequences A1 A2 A3 A4 A5… are divided into tuples A1 A2, A2 A3, A3 A4, A4 A5,… Then, TF represents the frequency of a tuple appearing in malicious software: TF = TN m / MTN, where TN mTW represents the number of times a tuple appears in malware, where MTN is the total number of tuples in malware. The design philosophy of TW is that if a tuple appears only in malware, it indicates that it is more representative of malicious behavior and should be given a higher weight. Conversely, if a tuple appears with roughly equal frequency in both malicious and non-malicious malware, it indicates that the malicious nature of its behavior is uncertain and should be given a lower weight. Therefore, this invention sets TW = (TN...) m / TN αll )*K, where TN αll This represents the total number of times a certain tuple appears. Considering the situation where a behavior has a low frequency but a high probability of being malicious, this invention sets a parameter K to amplify the weight. When TN... m / TN αll When the value is greater than 0.9, it means that there is a high probability that the tuple is associated with malicious behavior. K is set to 100 to amplify the weight; otherwise, it is set to 1. The larger the TF-TW value (TF×TW) of a tuple, the more frequently the tuple appears in malicious behavior and the less likely it is to appear in normal behavior. The TF-TW values ​​are sorted from largest to smallest, and the top 5% of TF-TW values ​​are extracted as the set of API tuples that can reflect malicious software behavior.

[0013] The aforementioned unlabeled API call sequence refers to the API call sequence extracted from software with unknown behavioral properties in an API call sequence dataset within a sandbox.

[0014] The labeled API call sequence refers to the API call sequence extracted from software with known behavioral properties in the API call sequence dataset within a sandbox.

[0015] The malware detection model includes:

[0016] (1) Embedding layer: The embedding layer includes a word embedding layer, a position embedding layer, and an API tuple information layer. The word embedding layer and the position embedding layer are each constructed through a fully connected layer; the word embedding layer converts each API in the input API call sequence into a vector of fixed dimensions; the position embedding layer is used to represent the vector of position information of each API call in the input sequence, and learns the order attribute of the input API call sequence.

[0017] The API tuple information layer treats the API tuple set as a dictionary. According to the dictionary, for any API call sequence, in addition to matching API tuples in the API tuple set, when both Tuple(n-1,n) and Tuple(n,n+1) are in the API tuple set, Tuple(n-1,n,n-1) is treated as an API tuple, where Tuple(n-1,n) represents the matched API tuple, and n-1 and n represent the API calls in the matched API tuple. Then, an API tuple matching matrix is ​​built based on the API call sequence and the matched API tuple set, where the positions of matched API tuples in the matrix are filled with 1s, and the positions of unmatched API tuples are filled with 0s.

[0018] The output of the embedding layer is the sum of the word embedding layer, the position embedding layer, and the API tuple information layer.

[0019] (2) Feature Extraction Layer: The feature extraction layer consists of 12 consecutive transformer encoders. A GELU activation function is used after each encoder. GELU has a smoother derivative, reducing the gradient vanishing problem during training. GELU also accelerates model convergence by introducing a sigmoid-like transformation in nonlinear transformations, allowing its output to fall within a wider range, thus speeding up convergence. In addition to residual connections and layer normalization operations common in deep neural networks, the encoder also includes a multi-head attention mechanism and a feedforward layer. The attention mechanism actually calculates correlations. First, X is input from the embedding layer... embedding A linear mapping is performed to generate Q, K, and V; where Q represents the query, K represents the key, and V represents the value. The calculation formula is Q = X. embedding *W Q K = X embedding *W K V = X embedding *W V , where X embedding It is the matrix after the embedding layer, W Q W K W V All are weight matrices. The formula for calculating the attention mechanism is: Where d k It is the dimension of each column of features in the key matrix. The multi-head attention mechanism works by using X... embeddingMultiple Q, K, and V are generated, and multiple Atttentions are generated. The output of the multi-head attention mechanism is the concatenation of multiple Atttentions and multiplied by a transformation matrix to keep the shape of the input and output matrices consistent. The feedforward layer consists of two linear mapping layers followed by the activation function ReLU.

[0020] (3) Linear Layer: In neural networks, a linear layer is a common type of network layer that implements a linear transformation between two layers of neurons. The purpose of a linear layer is to perform a linear transformation on the input data, thereby changing its distribution or dimension, facilitating subsequent nonlinear operations or other network layers. A linear layer can also be considered a feature transformation or dimensionality reduction technique. In this invention, the linear module is mainly used for dimensionality reduction to ultimately identify whether the software is malicious.

[0021] The technical advantages of this invention are as follows: Addressing the problem that existing pre-trained models cannot effectively learn the characteristics of malicious software behavior, this invention proposes TF-TW, which extracts API tuples that help malware detection models identify malicious behavior. By fully utilizing information about malicious software behavior, the invention trains malware detection models to enhance their learning of malicious behavior characteristics during both pre-training and fine-tuning stages. Compared to existing technologies, this invention enables malware detection models to fully learn these characteristics, accurately identify malicious behavior, and detect malware. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention;

[0023] Figure 2 This is a model diagram of the present invention;

[0024] Figure 3 This is a matrix diagram of malicious behavior information. Detailed Implementation

[0025] like Figure 1 As shown, this embodiment relates to a malware detection method based on a pre-trained model, including the following steps:

[0026] Step 1: Extract the API call sequence.

[0027] The collection consists of API call sequences extracted from running real sample software in a sandbox, including an unlabeled set of API call sequences for pre-training the malware detection model and a labeled set of API call sequences for fine-tuning.

[0028] In addition to the API call sequence, the executable file's runtime analysis results also include call name, call PID, and other information not used in this embodiment. Therefore, this embodiment first extracts the API call sequence from the analysis results and then removes the 'W' or 'A' at the end of each API call. The reason for removing the 'W' or 'A' at the end of the API calls is that this character only represents different coding styles and has no impact on semantic information.

[0029] Step 2: Generate a set of API tuples.

[0030] In the field of Natural Language Processing (NLP), TF-IDF (Term Frequency-Inverse Document Frequency) is a statistical method used to evaluate the importance of a word to a document in a corpus. TF measures the frequency of a word or phrase in the corpus, while IDF shows how much information that word or phrase provides. In the field of malware detection, research has also used TF-IDF to assign different weights to APIs or API runtime parameters to better identify malicious behavior. This embodiment borrows the idea of ​​TF-IDF and proposes TF-TW (Term Frequency-Term Weight) to evaluate which API tuples are helpful in detecting malicious behavior. Specifically, all API call sequences are first divided into binary tuples, for example, API call sequences A1 A2 A3 A4 A5… are divided into tuples A1A2, A2A3, A3A4, A4A5,… Then, TF represents the frequency of a tuple appearing in malware: TF = TN m / MTN, where TN m TW represents the number of times a tuple appears in malware, where MTN is the total number of tuples in malware. The design philosophy of TW is that if a tuple appears only in malware, it indicates that it is more representative of malicious behavior and should be given a higher weight. Conversely, if a tuple appears with roughly equal frequency in both malicious and non-malicious malware, it indicates that the malicious nature of its behavior is uncertain and should be given a lower weight. Therefore, this invention sets TW = (TN...) m / TN αll )*K, where TN αll This represents the total number of times a certain tuple appears. Considering the situation where a behavior has a low frequency but a high probability of being malicious, this invention sets a parameter K to amplify the weight. When TN... m / TN αllWhen the value is greater than 0.9, it means that there is a high probability that the tuple is associated with malicious behavior. K is set to 100 to amplify the weight; otherwise, it is set to 1. The larger the TF-TW value (TF×TW) of a tuple, the more frequently the tuple appears in malicious behavior and the less likely it is to appear in normal behavior. The TF-TW values ​​are sorted from largest to smallest, and the top 5% of TF-TW values ​​are extracted as the set of API tuples that can reflect malicious software behavior.

[0031] Step 3: The malware detection model is first pre-trained using a publicly available text dataset from Wikipedia. Then, it is further pre-trained using unlabeled API call sequences. Finally, the unlabeled API call sequences are input into the malware detection model to train a masked language model (MLM).

[0032] By designing a pre-defined network structure for language modeling tasks, and then extracting linguistic knowledge from a large amount of unlabeled natural language text and encoding it into the network structure, these prior linguistic features can significantly supplement the features of a specific task when labeled data is limited, thereby enhancing the model's generalization ability. Training a neural network model requires a large number of samples to converge, but obtaining malware samples and extracting their runtime API call sequences is not an easy task. The biggest advantage of pre-trained models is the reusability of low-level features, which effectively solves the problem of insufficient samples.

[0033] like Figure 2 As shown, the malware detection model includes an embedding layer, a feature extraction layer, and a linear layer.

[0034] The embedding layer consists of a word embedding layer, a location embedding layer, and an API tuple information layer.

[0035] The word embedding layer and the position embedding layer are each composed of fully connected layers; the word embedding layer converts each API in the input API call sequence into a vector of fixed dimensions; the position embedding layer is used to represent the vector of position information of each API call in the input sequence, and learns the order attribute of the input API call sequence.

[0036] (1) The role of the word embedding layer is to convert each API in the input API call sequence into a vector of fixed dimensions, which is 768 dimensions in this embodiment.

[0037] (2) The position embedding layer allows the model to learn the order attributes of the input.

[0038] (3) The API tuple information layer incorporates API tuple information into the malware detection model, enabling the model to learn the relationships between API tuples and thus better identify malware. Specifically, the previously extracted API tuple set is treated as a dictionary. Based on this dictionary, for any API call sequence, besides matching API tuples in the API tuple set, when both Tuple(n,n-1) and Tuple(n,n+1) are in the API tuple set, Tuple(n-1,n,n-1) is considered an API tuple. Tuple(n-1,n) represents a matched API tuple, and n-1 and n represent the API calls within the matched API tuple. Then, an API tuple matching matrix is ​​constructed based on the API call sequence and the matched API tuple set. The positions of matched API tuples in the matrix are filled with 1s, and the unmatched positions are filled with 0s.

[0039] The output of the embedding layer is the sum of the word embedding layer, the position embedding layer, and the API tuple information layer. Figure 3 This is a flowchart for generating API tuple information.

[0040] The feature extraction layer consists of 12 consecutive transformer encoders. A GELU activation function is used after each encoder. GELU has a smoother derivative, reducing the gradient vanishing problem during training. GELU also accelerates model convergence by introducing a sigmoid-like transformation in nonlinear transformations, allowing its output to fall within a wider range, thus contributing to faster convergence. In addition to residual connections and layer normalization operations common in deep neural networks, the encoder also includes a multi-head attention mechanism and a feedforward layer. The attention mechanism essentially calculates correlations. First, the input X is taken from the embedding layer... embedding A linear mapping is performed to generate Q, K, and V; where Q represents the query, K represents the key, and V represents the value. The calculation formula is Q = X. embedding *W Q K = X embedding *W k V = X embedding *W V , where X embedding It is the matrix after the embedding layer, W Q W K W V All are weight matrices. The formula for calculating the attention mechanism is: Where d k It is the dimension of each column of features in the key matrix. The multi-head attention mechanism is achieved through X. embeddingMultiple Q, K, and V are generated, and multiple Atttentions are generated. The output of the multi-head attention mechanism is the concatenation of multiple Atttentions and multiplied by a transformation matrix to keep the shape of the input and output matrices consistent. The feedforward layer consists of two linear mapping layers followed by the activation function ReLU.

[0041] Linear layers implement linear transformations between two layers of neurons. The purpose of linear layers is to linearly transform input data, thereby changing its distribution or dimensionality, facilitating subsequent nonlinear operations or other network layers. Linear layers are considered a feature transformation or dimensionality reduction technique. This embodiment uses linear layer dimensionality reduction to identify whether software is malware.

[0042] This embodiment employs a masked language model (MLM) as the pre-training task. MLM randomly masks some tokens in a sequence, and then trains the malware detection model to guess these masked tokens. Through this training, the malware detection model can learn not only the relationships between individual API functions in an API call sequence, but also the relationships between API combinations through API tuple information. Research shows that domain-adaptive pre-training and task-adaptive pre-training can further improve the performance of the pre-trained model. Therefore, this embodiment uses the parameters of a pre-trained model trained on a general knowledge corpus and a computer science domain corpus from Hugging Face for initialization, and further pre-trains the malware detection model using unlabeled API call sequences.

[0043] Step 4: Use labeled API call sequences to fine-tune the model to help the malware detection model fit and achieve binary classification, ultimately realizing malware detection.

[0044] Based on specific practical experiments, the implementation was carried out under the following environment settings: CPU Intel Core i513@2.20GHz, GPU NVIDIA RTX3060 12GB, operating system Windows 10, and simulation environment Python.

[0045] This embodiment evaluates the efficiency and effectiveness of the algorithm by conducting experiments on a real malware API call sequence dataset. This dataset consists of two parts: the first part contains 40,000 unlabeled API call sequences for pre-training the malware detection model; the second part contains 40,000 labeled API call sequences for fine-tuning the pre-trained malware detection model. The second part of the dataset contains roughly half malware API call sequences and half benign software API call sequences. This dataset is derived from a third-party public release on GitHub, and all data comes from software samples running in a sandbox, extracted from the run reports. In this embodiment, MalTBERT is used in the experiments, and the results are shown in Table 1, in percentage terms.

[0046] Table 1

[0047]

[0048] The experiment used Accuracy, Precision, Recall, and F1-Score as model evaluation metrics, and the calculation methods are as follows:

[0049]

[0050]

[0051]

[0052]

[0053] TP (True Positive) represents classifying malware as malware, TN (True Negative) represents classifying benign software as benign software, FN (False Negative) represents classifying malware as benign software, and FP (False Positive) represents classifying benign software as malware.

[0054] Experimental results show that MalTBERT exhibits high evaluation metrics on the test set. As shown in Table 1, both the pre-trained models BERT and RoBERTa (a pre-trained language model based on the Transformer architecture) achieved evaluation metrics above 95% on the test set. This result demonstrates that pre-trained models can achieve excellent malware detection capabilities when trained on datasets based on API call sequences. MalTBERT demonstrates better detection capabilities than BERT and RoBERTa, achieving an ultra-high detection rate of 98.9% in both Accuracy and F1-Score, and an even higher Recall of 99.8%. This indicates that API tuple information is highly effective for the model in identifying malware.

[0055] This embodiment compares MalTBERT with four commonly used machine learning methods—KNN (K-Nearest Neighbors), LR (Logistic Regression), Bayes, and SVM (Support Vector Machine)—and two deep learning methods, including the paper's Convolutional Neural Network (Text-CNN) and Bidirectional Long Short-Term Memory Network (BiLSTM), on a test dataset in terms of accuracy, precision, recall, and F1-score. As shown in Table 1, MalTBERT outperforms other deep learning and machine learning models. For example, MalTBERT achieves an F1-score of 98.92%, while the highest score among other models is 96.18% for Text-CNN. This method outperforms existing methods in all four metrics: accuracy, precision, recall, and F1-score.

[0056] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A malware detection method based on a malicious behavior-enhanced pre-trained model, characterized in that, Includes the following steps: Step 1: Extract API call sequences from the API call sequence dataset; Step 2: Construct a corpus from all API call sequences and generate a set of API tuples reflecting malicious software behavior according to statistical rules; Step 3: Pre-train the malware detection model using a public dataset, and then continue pre-training using an unlabeled API call sequence; The malware detection model comprises a sequentially connected embedding layer, a feature extraction layer, and a linear layer; The embedding layer includes a word embedding layer, a location embedding layer, and an API tuple information layer; The feature extraction layer consists of 12 consecutive transformer encoders, with the GELU activation function applied after each encoder. The encoder also includes a multi-head attention mechanism and a feedforward layer, which consists of two linear mapping layers followed by the ReLU activation function. The word embedding layer and the position embedding layer are each composed of fully connected layers; the word embedding layer converts each API in the input API call sequence into a vector of fixed dimensions; the position embedding layer is used to represent the vector of position information of each API call in the input sequence, and learns the order attribute of the input API call sequence. The API tuple information layer treats the API tuple set as a dictionary. Based on this dictionary, for any API call sequence, besides matching API tuples in the API tuple set, when the first API call occurs... and When all are in the API tuple set, Consider it as an API tuple, where This indicates the API tuples that have been matched. and This represents the API calls in the matched API tuples; then, an API tuple matching matrix is ​​built based on the API call sequence and the set of matched API tuples, where the positions of the matched API tuples in the matrix are filled with 1s and the positions of the unmatched API tuples are filled with 0s. The output of the embedding layer is the sum of the word embedding layer, the position embedding layer, and the API tuple information layer; Step 4: Fine-tune the malware detection model using a labeled API call sequence, and perform binary classification through linear dimensionality reduction to complete malware detection.

2. The malware detection method based on a malicious behavior enhancement pre-trained model according to claim 1, characterized in that, The API call sequence dataset mentioned in step 1 is a set of API call sequences extracted from real sample software running in a sandbox. It includes an unlabeled API call sequence set for pre-training the malware detection model and a labeled API call sequence set for fine-tuning.

3. The malware detection method based on a malicious behavior enhancement pre-trained model according to claim 2, characterized in that, The specific process of generating the set of API tuples reflecting malicious software behavior according to statistical rules in step 2 is as follows: 2.1 Divide all API call sequences into tuples; 2.2 Definition This represents the frequency with which a tuple appears in malware: ,in This represents the number of times a particular tuple appears in malware. This represents the total number of tuples in the malware. definition ,in This represents the total number of times a given tuple appears. ; 2.

3. Define TF-TW value as The TF-TW values ​​are sorted from largest to smallest, and the top 5% of TF-TW values ​​are extracted as the API tuple set reflecting malicious software behavior.

4. The malware detection method based on a malicious behavior enhancement pre-trained model according to claim 3, characterized in that, In steps 3 and 4, the unlabeled API call sequence refers to the API call sequence extracted from the API call sequence dataset by software with unknown behavioral properties in a sandbox. The labeled API call sequence refers to the API call sequence extracted from software with known behavioral properties in the API call sequence dataset within a sandbox.

5. The malware detection method based on a malicious behavior enhancement pre-trained model according to claim 4, characterized in that, In step 3, the step of using unlabeled API call sequences for further pre-training specifically involves inputting the unlabeled API call sequences into the malware detection model and training the malware detection model using a masked language model (MLM).

Images