A dynamic malware detection method based on enhanced semantic API sequence features
By using the RoBERTa model and multi-head attention mechanism, combined with variational autoencoder and gating mechanism, enhanced semantic API sequence features are generated, which solves the problems of insufficient sequence semantic mining and model fragmentation in existing malware detection technologies, and improves detection efficiency and accuracy.
Patent Information
- Application Number
- CN202411140093.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-20
AI Technical Summary
Existing malware detection methods suffer from insufficient sequence semantic mining and API sequence fragmentation caused by fine-tuning of pre-trained models when facing unknown or novel malware, affecting detection efficiency and accuracy.
The robustly optimized BERT pre-trained model RoBERTa is used to obtain contextual semantic information. Combined with a multimodal weight control module with variational autoencoder and gating mechanism, enhanced semantic API sequence features are constructed through multi-head attention mechanism to capture API call behavior patterns and generate global feature representations.
It improves the detection generalization ability of new and variant malware, reduces the complexity of feature processing and the number of model parameters, and enhances the accuracy and efficiency of detection.
Smart Images

Figure CN118656827B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of malware detection, and in particular to a dynamic malware detection method based on enhanced semantic API sequence features. Background Art
[0002] With the popularization of computers and networks, the number and types of malware are increasing. Researchers have proposed various static and dynamic analysis methods to detect malware. Static analysis methods do not need to execute malware samples. Instead, they analyze static features such as strings, function calls, and API calls by studying source code or binary files. This method can avoid the risk of infection caused by running malware and takes up fewer resources. However, malware developers use obfuscation techniques such as code encryption, program instruction reordering, and dead code insertion to circumvent static analysis, which causes static analysis methods to consume more time to parse the executable file structure, thereby affecting the efficiency and accuracy of the analysis. In addition, static analysis relies on known features or signatures, so it has limitations when dealing with unknown or new malware. Dynamic analysis methods load executable programs into main memory and monitor their actual running behaviors for analysis. Dynamic analysis monitors the behavior of executable programs such as registry key changes, URL access, and API calls during runtime. Through the analysis of behavioral patterns, it effectively responds to the threat of zero-day malware. At the same time, malware from the same family often exhibits the same or highly similar behaviors, which can be described by the API call sequence at runtime. However, most existing methods rely more on parameter information in the API calling process to build detection and classification systems, which leads to excessive reliance on prior knowledge in feature processing, large number of model parameters and high computational complexity, while ignoring the dynamic API calling sequence in the context and failing to fully mine complex semantics. Summary of the invention
[0003] Purpose of the invention: The purpose of the present invention is to provide a dynamic malware detection method based on enhanced semantic API sequence features to solve the problems in the prior art of insufficient sequence semantic mining and API sequence fragmentation caused by fine-tuning of pre-trained models.
[0004] Technical solution: The present invention provides a dynamic malware detection method based on enhanced semantic API sequence features, comprising the following steps:
[0005] (1) Upload the executable file to the sandbox to obtain a dynamic behavior report containing the API sequence and its parameters; randomly select dynamic behavior reports of non-repetitive malware and benign software as experimental data, and divide them into training sets and test sets; and perform preprocessing;
[0006] (2) Use the robustly optimized BERT pre-training model RoBERTa to obtain contextual semantic information;
[0007] (3) The API call frequency is sampled and encoded through a variational autoencoder to obtain a universal representation of global API call features, thereby capturing the behavioral patterns of API calls in the system;
[0008] (4) A multimodal weight control module based on a gating mechanism adjusts the weights of each modal feature, allowing semantic features and global features to interact and generate semantically enhanced API sequence features;
[0009] (5) A multi-head attention mechanism is used to build a detection model based on enhanced semantic API sequence features.
[0010] Furthermore, the preprocessing of step (1) is specifically as follows: a dictionary is constructed using the public API captured by Cuckoo Sandbox, the Windows system API publicly available from Microsoft, and the API included in the training set, thereby converting the API sequence into a vector representation; the collected dynamic behavior reports are preprocessed using a data preprocessing algorithm to remove the redundant information contained therein.
[0011] Furthermore, step (2) is as follows: the length of the API sequence is normalized to the maximum input length of RoBERTa, 512, and the sequence is obtained.
[0012] ;
[0013] And use attention mask marking to fill in symbols; train the API sequence to obtain the semantic feature map between API calls, the formula is as follows:
[0014] ;
[0015] in, The API call sequence s extracted by the RoBERTa model corresponds to the semantic features of the API;
[0016] The API semantic feature map C is mapped to the information space through a dense layer:
[0017] ;
[0018] in, is the sigmoid activation function, and They are the weight matrix and bias vector of the neural network respectively; It is the mapping matrix of the semantic features of API sequence.
[0019] Furthermore, step (3) includes the following steps:
[0020] (31) Extracting the global frequency features of API calls in the training set, that is, counting the number of times a single API appears in malicious or benign programs and dividing it by the total number for normalization; inputting the frequency matrix corresponding to the API call sequence of each software into the variational autoencoder VAE for encoding, mapping the API frequency features of the executable program to the vector representation of the latent space, and extracting and mining the global features of API calls;
[0021] (32) The statistical vector generated by the API call sequence of all executable programs in the training set is recorded as ;in, is the statistical vector of different program API sequences, with a total of N; the VAE encoder has a statistical vector for the input Generating latent variables The probability distribution of ,in, It is the encoder parameter set that contains all the weights and biases that need to be learned during the encoding process; the latent variable values are sampled using the reparameterization method ;
[0022] (33) The latent variable value obtained by sampling Map it back to the input space through the decoder for back propagation to obtain the reconstructed probability distribution ; The loss function formula is as follows:
[0023] ;
[0024] Among them, the expected Used to measure the ability to generate data, is a given input Latent variables The posterior distribution of ; KL The term measures the posterior distribution With the prior distribution The difference between them is used to ensure that the learned potential API representation is close to the prior distribution; by minimizing the two loss terms, the potential API global representation Closer to the prior distribution , so that the encoder generates a latent vector that can represent the global feature information of the API;
[0025] (34) The global characteristic distribution of API sequences is fitted using the reparameterization method, and the formula is as follows:
[0026] ;
[0027] in, is the multivariate Gaussian prior distribution of API global features, represents the mean of the distribution, represents the standard deviation, represents the covariance matrix.
[0028] Furthermore, step (4) is as follows: the latent vector representation of the global information of the executable program obtained by the VAE encoder is Project it into an information space of the same dimension as the semantic feature to obtain a vector representation of the global feature ; Use the AdaGate function of the gating mechanism to fuse semantic information and global information Construct a multimodal feature weight control module to dynamically adjust the feature weights between modalities; use global features to obtain API sequence features with enhanced semantics , the formula is as follows:
[0029] ;
[0030] in represents the activation function, stands for element-wise product; Is semantic information Probabilistic representation obtained through the linear layer;
[0031] By using the Valve function in the gating mechanism to dynamically adjust the global feature information, the global information is removed The formula for API calls with lower classification confidence is as follows:
[0032] For each API feature ,
[0033] ;
[0034] Among them, the Valve function is a global feature information dynamic adjustment function. is the confidence threshold, used to adjust the global information API Features Specifically, if = 0, all global information will be discarded; if = 0.5 , all global information is accepted.
[0035] Furthermore, step (5) includes the following steps:
[0036] (51) Through multimodal fusion features Generate initial query, key and value; where Q is the query matrix, K is the key matrix, and V is the value matrix;
[0037] (52) Generate different feature heads using linear layer transformation , the formula is as follows:
[0038] ;
[0039] Among them, h represents the hth feature head, They are the query weight matrix, key weight matrix, and value weight matrix of the h-th head respectively;
[0040] (53) Multiply the matrices Q, K, and V by each different feature head to obtain different attention weights for the API sequence features, as follows:
[0041] ;
[0042] in, , and Represent the query, key and value representations of different feature heads respectively;
[0043] (54) Calculate the attention results of each head separately, the formula is as follows:
[0044] ;
[0045] Among them, Softmax is the activation function, Attention is the attention result of the i-th head;
[0046] (55) Concatenate the attention results of all attention heads and multiply them by the weight matrix To obtain the feature representation of the enhanced semantic API sequence, the formula is as follows:
[0047] ;
[0048] Among them, Concat(⋅) means concatenating the attention results of all heads; the concatenated attention results are transformed through a linear weight matrix , and get the final multi-head attention output ;
[0049] (56) The enhanced semantic API sequence features output by the multi-head attention are reduced in dimension through the pooling layer and then input into the fully connected neural network to build a malware detection model.
[0050] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the RoBERTa model is used to mine semantic features on the original API sequence. Compared with BERT, RoBERTa removes the context prediction task in the pre-training process, and can process the API call sequence of a single malware as a whole, avoiding the information loss and reduced computational efficiency caused by segmenting the API call sequence into multiple sentences. At the same time, RoBERTa uses a dynamic masking method to help the model better understand the contextual information of the API sequence, thereby more accurately capturing the semantic information in the API sequence. In addition, in order to further capture API dependencies, a gating mechanism is proposed to dynamically adjust the input weights of semantic and global features in the detection model, forming semantically enhanced API sequence features to reduce the complexity of feature processing and the amount of model parameters, further improving the generalization ability of the solution for detecting new and variant malware. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 This is a diagram of the malware detection system architecture of the present invention;
[0052] Figure 2 for semantic feature generation of the present invention;
[0053] Figure 3 It is a structural diagram of the detection model of the present invention. DETAILED DESCRIPTION
[0054] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0055] like Figure 1 As shown, an embodiment of the present invention provides a dynamic malware detection method based on enhanced semantic API sequence features, comprising the following steps:
[0056] (1) Upload the executable file to the sandbox to obtain a dynamic behavior report containing the API sequence and its parameters; randomly select non-repetitive dynamic behavior reports of malware and benevolent software as experimental data, and divide them into training sets and test sets; and perform preprocessing; the preprocessing is as follows: use the public API captured by the Cuckoo Sandbox, the Windows system API publicly available from Microsoft, and the API included in the training set to build a dictionary, thereby converting the API sequence into a vector representation; preprocess the collected dynamic behavior reports using a data preprocessing algorithm to remove the redundant information they contain.
[0057] (2) Use the robustly optimized BERT pre-training model RoBERTa to obtain contextual semantic information; Figure 2 As shown, the details are as follows: The length of the API sequence is normalized to the maximum input length of RoBERTa, 512, and the sequence is obtained. ;
[0058] And use attention mask marking to fill in symbols; train the API sequence to obtain the semantic feature map between API calls, the formula is as follows:
[0059] ;
[0060] in, The API call sequence s extracted by the RoBERTa model corresponds to the semantic features of the API;
[0061] The API semantic feature map C is mapped to the information space through a dense layer:
[0062] ;
[0063] in, is the sigmoid activation function, and They are the weight matrix and bias vector of the neural network respectively; It is the mapping matrix of the semantic features of API sequence.
[0064] (3) Sampling and encoding the API call frequency through a variational autoencoder to obtain a general representation of the global API call features, thereby capturing the behavioral patterns of API calls in the system; including the following steps:
[0065] (31) Extracting the global frequency features of API calls in the training set, that is, counting the number of times a single API appears in malicious or benign programs and dividing it by the total number for normalization; inputting the frequency matrix corresponding to the API call sequence of each software into the variational autoencoder VAE for encoding, mapping the API frequency features of the executable program to the vector representation of the latent space, and extracting and mining the global features of API calls;
[0066] (32) The statistical vector generated by the API call sequence of all executable programs in the training set is recorded as ;in, is the statistical vector of different program API sequences, with a total of N; the VAE encoder has a statistical vector for the input Generating latent variables The probability distribution of ,in, It is the encoder parameter set that contains all the weights and biases that need to be learned during the encoding process; the latent variable values are sampled using the reparameterization method ;
[0067] (33) The latent variable value obtained by sampling Map it back to the input space through the decoder for back propagation to obtain the reconstructed probability distribution ; The loss function formula is as follows:
[0068] ;
[0069] Among them, the expected Used to measure the ability to generate data, is a given input Latent variables The posterior distribution of ; KL The term measures the posterior distribution With the prior distribution The difference between them is used to ensure that the learned potential API representation is close to the prior distribution; by minimizing the two loss terms, the potential API global representation Closer to the prior distribution , so that the encoder generates a latent vector that can represent the global feature information of the API;
[0070] (34) The global characteristic distribution of API sequences is fitted using the reparameterization method, and the formula is as follows:
[0071] ;
[0072] in, is the multivariate Gaussian prior distribution of API global features, represents the mean of the distribution, represents the standard deviation, represents the covariance matrix.
[0073] (4) The multimodal weight control module based on the gating mechanism adjusts the weights of each modal feature to enable the interaction between semantic features and global features to generate semantically enhanced API sequence features; the latent vector representation of the global information of the executable program obtained by the VAE encoder is Project it into an information space of the same dimension as the semantic feature to obtain a vector representation of the global feature ; Use the AdaGate function of the gating mechanism to fuse semantic information and global information Construct a multimodal feature weight control module to dynamically adjust the feature weights between modalities; use global features to obtain API sequence features with enhanced semantics , the formula is as follows:
[0074] ;
[0075] in represents the activation function, stands for element-wise product; Is semantic information Probabilistic representation obtained through the linear layer;
[0076] By using the Valve function in the gating mechanism to dynamically adjust the global feature information, the global information is removed The formula for API calls with lower classification confidence is as follows:
[0077] For each API feature ,
[0078] ;
[0079] Among them, the Valve function is a global feature information dynamic adjustment function. is the confidence threshold, used to adjust the global information API Features Specifically, if = 0, all global information will be discarded; if = 0.5 , all global information is accepted.
[0080] (5) A multi-head attention mechanism is used to build a detection model based on enhanced semantic API sequence features. Figure 3 As shown, the following steps are included:
[0081] (51) Through multimodal fusion features Generate initial query, key and value; where Q is the query matrix, K is the key matrix, and V is the value matrix;
[0082] (52) Generate different feature heads using linear layer transformation , the formula is as follows:
[0083] ;
[0084] Among them, h represents the hth feature head, They are the query weight matrix, key weight matrix, and value weight matrix of the h-th head respectively;
[0085] (53) Multiply the matrices Q, K, and V by each different feature head to obtain different attention weights for the API sequence features, as follows:
[0086] ;
[0087] in, , and Represent the query, key and value representations of different feature heads respectively;
[0088] (54) Calculate the attention results of each head separately, the formula is as follows:
[0089] ;
[0090] Among them, Softmax is the activation function, Attention is the attention result of the i-th head;
[0091] (55) Concatenate the attention results of all attention heads and multiply them by the weight matrix To obtain the feature representation of the enhanced semantic API sequence, the formula is as follows:
[0092] ;
[0093] Among them, Concat(⋅) means concatenating the attention results of all heads; the concatenated attention results are transformed through a linear weight matrix , and get the final multi-head attention output ;
[0094] (56) The enhanced semantic API sequence features output by the multi-head attention are reduced in dimension through the pooling layer and then input into the fully connected neural network to build a malware detection model.
Claims
1. A dynamic malware detection method based on enhanced semantic API sequence features, characterized in that: The following steps are involved: (1) Upload the executable file to the sandbox to obtain a dynamic behavior report containing the API sequence and its parameters; randomly select dynamic behavior reports of non-repetitive malware and benign software as experimental data, and divide them into training sets and test sets; and perform preprocessing; (2) Use the robustly optimized BERT pre-trained model RoBERTa to obtain contextual semantic information; specifically: normalize the length of the API sequence to the maximum input length of RoBERTa, 512, and obtain the sequence ; And use attention mask marking to fill in symbols; train the API sequence to obtain the semantic feature map between API calls, the formula is as follows: ; in, The API call sequence s extracted by the RoBERTa model corresponds to the semantic features of the API; The API semantic feature map C is mapped to the information space through a dense layer: ; in, is the sigmoid activation function, and They are the weight matrix and bias vector of the neural network respectively; is the mapping matrix of the semantic features of the API sequence; (3) Sampling and encoding the API call frequency through a variational autoencoder to obtain a general representation of the global API call features, thereby capturing the behavioral patterns of API calls in the system; including the following steps: (31) Extracting the global frequency features of API calls in the training set, that is, counting the number of times a single API appears in malicious or benign programs and dividing it by the total number for normalization; inputting the frequency matrix corresponding to the API call sequence of each software into the variational autoencoder VAE for encoding, mapping the API frequency features of the executable program to the vector representation of the latent space, and extracting and mining the global features of API calls; (32) The statistical vector generated by the API call sequence of all executable programs in the training set is recorded as ;in, is the statistical vector of different program API sequences, with a total of N; the VAE encoder has a statistical vector for the input Generating latent variables The probability distribution of ,in, It is the encoder parameter set that contains all the weights and biases that need to be learned during the encoding process; the latent variable values are sampled using the reparameterization method ; (33) The latent variable value obtained by sampling Map it back to the input space through the decoder for back propagation to obtain the reconstructed probability distribution ; The loss function formula is as follows: ; Among them, the expected Used to measure the ability to generate data, is a given input Latent variables The posterior distribution of ; KL The term measures the posterior distribution With prior distribution The difference between them is used to ensure that the learned potential API representation is close to the prior distribution; by minimizing the two loss terms, the potential API global representation Closer to the prior distribution , so that the encoder generates a latent vector that can represent the global feature information of the API; (34) The global characteristic distribution of API sequences is fitted using the reparameterization method, and the formula is as follows: ; in, is the multivariate Gaussian prior distribution of API global features, represents the mean of the distribution, represents the standard deviation, represents the covariance matrix; (4) The multimodal weight control module based on the gating mechanism adjusts the weights of each modal feature to enable the interaction between semantic features and global features to generate semantically enhanced API sequence features; specifically, the latent vector representation of the executable program global information obtained by the VAE encoder is Project it into an information space of the same dimension as the semantic feature to obtain a vector representation of the global feature ; Use the AdaGate function of the gating mechanism to fuse semantic information and global information Construct a multimodal feature weight control module to dynamically adjust the feature weights between modalities; use global features to obtain API sequence features with enhanced semantics , the formula is as follows: ; in represents the activation function, stands for element-wise product; Is semantic information Probabilistic representation obtained through the linear layer; By using the Valve function in the gating mechanism to dynamically adjust the global feature information, the global information is removed The formula for API calls with lower classification confidence is as follows: For each API feature , ; Among them, the Valve function is a global feature information dynamic adjustment function. is the confidence threshold, used to adjust the global information API Features Specifically, if = 0, all global information will be discarded; if =0.5, then accept all global information; (5) A multi-head attention mechanism is used to build a detection model based on enhanced semantic API sequence features; the following steps are included: (51) Through multimodal fusion features Generate initial query, key and value; where Q is the query matrix, K is the key matrix, and V is the value matrix; (52) Generate different feature heads using linear layer transformation , the formula is as follows: ; Among them, h represents the hth feature head, They are the query weight matrix, key weight matrix, and value weight matrix of the h-th head respectively; (53) Multiply the matrices Q, K, and V by each different feature head to obtain different attention weights for the API sequence features, as follows: ; in, , and Represent the query, key and value representations of different feature heads respectively; (54) Calculate the attention results of each head separately, the formula is as follows: ; Among them, Softmax is the activation function, Attention is the attention result of the i-th head; (55) Concatenate the attention results of all attention heads and multiply them by the weight matrix To obtain the feature representation of the enhanced semantic API sequence, the formula is as follows: ; Among them, Concat(⋅) means concatenating the attention results of all heads; the concatenated attention results are transformed through a linear weight matrix , and get the final multi-head attention output ; (56) The enhanced semantic API sequence features output by the multi-head attention are reduced in dimension through the pooling layer and then input into the fully connected neural network to build a malware detection model.
2. A dynamic malware detection method based on enhanced semantic API sequence features according to claim 1, characterized in that: Step (1) preprocessing is as follows: a dictionary is constructed using the public API captured by Cuckoo Sandbox, the Windows system API publicly available from Microsoft, and the API included in the training set, thereby converting the API sequence into a vector representation; the collected dynamic behavior reports are preprocessed using a data preprocessing algorithm to remove the redundant information contained therein.
Citation Information
Patent Citations
Industrial internet-oriented semantic enhanced malicious software detection method
CN117521066A
Intelligent malicious software detection method oriented to semantic analysis
CN117688560A
