API semantic vectorization Transform malicious code detection method and system

Abstract

The invention discloses an API (Application Program Interface) semantic vectorization Transform malicious code detection method and system. The method comprises the steps of obtaining a malicious code data set based on a PE file, extracting an API sequence and an MD5 value of a malicious code in the malicious code data set, and determining a malicious code family label according to the MD5 value; constructing a FastText word embedding model based on negative sampling optimization, and generating a word vector from the API sequence of the malicious code based on the FastText word embedding model; the method comprises the following steps: performing oversampling processing on a malicious code API vector by using a BorederLine-1SMOTE algorithm, and performing data balance by synthesizing minority class samples of a class boundary; constructing a Transform malicious code feature extraction model, and performing feature extraction and semantic analysis on the malicious code of the malicious code API vector; an average pooling layer and a full connection layer are added to the Transform malicious code feature extraction model, a malicious code detection model is constructed, and malicious code classification detection is achieved. According to the scheme, the malicious code recognition capability and detection performance are improved.

Description

Technical Field

[0001] The present invention belongs to the field of cyberspace security and artificial intelligence, and in particular relates to an API semantic vectorization Transformer malicious code detection method and system. Background Art

[0002] With the rapid development of information technology, malicious code has posed a huge threat to people's normal work and life, and brought a series of significant losses and adverse effects to economic development. Therefore, malicious code detection has become one of the most critical issues in the field of computer security.

[0003] Malicious code is a computer program or script designed to perform unauthorized harmful actions. These actions may include, but are not limited to, data theft, system damage, misuse of computer resources, and the spread of other malicious software. In the early days, malicious code mainly existed in the form of viruses and worms, which spread by infecting files or self-replicating. These malicious codes are usually relatively simple, and their main purpose is to cause damage to the system or self-propagate.

[0004] With the development of the Internet, the targets of malicious code have gradually turned to personal privacy and commercial interests. Spyware and adware have become widely spread types of malicious code, which make profits by tracking users' online activities and displaying targeted advertisements. The emergence of ransomware is an important turning point. This type of malicious code encrypts users' files and extorts ransom from victims for decryption keys, which has become one of the main ways for hackers to make profits. With the popularity of mobile devices, malicious code has begun to affect mobile platforms. The continuous increase in malicious applications and viruses has brought new threats to mobile devices. This threat is not limited to individual users, but also poses a challenge to the security of enterprises and organizations. Therefore, protecting computers and mobile devices from malicious code attacks has become a vital task.

[0005] With the development of obfuscation and packaging technology, many variants of malicious code have emerged. This situation has caused traditional detection methods to become inefficient and ineffective in discovering and identifying these variants. When faced with a large amount of malicious code, existing malicious code detection models have problems such as being easily confused by similar codes, failing to fully consider the semantic relationship of malicious code API context, and having reduced accuracy in representing low-frequency malicious code. Summary of the invention

[0006] In order to address the deficiencies in the prior art, the present invention provides an API semantic vectorization Transformer malicious code detection method and system to solve the technical problem that the malicious code detection method has reduced detection capability when facing first-appearing malicious code variants such as polymorphic attacks and zero-day attacks.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions.

[0008] The present invention first discloses an API semantic vectorization Transformer malicious code detection method, which includes the following steps:

[0009] Step 1: Obtain a malicious code data set based on a PE file, extract the API sequence and MD5 value of the malicious code in the malicious code data set, and determine the malicious code family label according to the MD5 value;

[0010] Step 2: construct a FastText word embedding model based on negative sampling optimization, and generate a word vector for the API sequence of the malicious code based on the FastText word embedding model;

[0011] Step 3: Use the BorederLine-1 SMOTE algorithm to oversample the malicious code API vectors and balance the data by synthesizing minority class samples at the boundary of the class.

[0012] Step 4: construct a Transformer malicious code feature extraction model to perform feature extraction and semantic analysis on the malicious code of the malicious code API vector;

[0013] Step 5: Add an average pooling layer and a fully connected layer to the Transformer malicious code feature extraction model to build a malicious code detection model to achieve malicious code classification detection.

[0014] The present invention further includes the following preferred embodiments:

[0015] The extracting the API sequence and MD5 value of the malicious code in the malicious code data set, and determining the malicious code family label according to the MD5 value, further includes:

[0016] Step 1.1: Download a malicious code benchmark dataset based on the PE file format, and extract the API call sequence and MD5 features of the malicious code in the malicious code benchmark dataset based on the PE format; the PE file of each malicious code sample in the dataset contains SHA256, MD5, tags, header files, release time, and API functions. Use Python code to extract the MD5 value and API function in the PE header file of each malicious code sample;

[0017] Step 1.2: Distinguish the malicious code family to which each malicious code sample belongs; input the unique MD5 value corresponding to the malicious code API function into the virus scanning website, determine the malicious code family through the virus scanning engine of the virus scanning website, and save the obtained malicious code and malicious code family labels as a .csv file.

[0018] The constructing of a FastText word embedding model based on negative sampling optimization, and generating a word vector from the API sequence of the malicious code based on the FastText word embedding model, further includes:

[0019] Step 2.1: Segment the API sequence of the malicious code and generate a dictionary; use the jieba word segmentation tool to segment the data set, remove irrelevant characters before and after, segment according to the API sequence and API call function, generate a word list, and then use the tokenizer tool to convert the segmented text data into discrete tokens;

[0020] Step 2.2: Build the input layer of the FastText word embedding model based on negative sampling optimization to obtain the embedding vectors of the n-gram subwords and special subwords of each API function;

[0021] Step 2.3: Construct the hidden layer of the FastText word embedding model based on negative sampling optimization, and perform weighted average of the embedding vectors of all subwords;

[0022] Step 2.4: Construct the output layer of the FastText word embedding model based on negative sampling optimization and output the malicious code API vector.

[0023] The construction of the FastText word embedding model input layer based on negative sampling optimization, obtaining the embedding vectors of the n-gram subwords and special subwords of each API function, further includes:

[0024] Character-level n-gram information is used to capture the sequential relationship between characters. Each API function in the input sequence is processed as n-gram subwords, and each central word is represented as a collection of subwords. "<" and ">" symbols are added to the first position of the word to separate the words to distinguish prefix subwords from suffix subwords. The n-gram subword information of an API function is extracted, and the word as a whole is regarded as a special subword. All n-gram subwords and special subwords are represented by embedding vectors.

[0025] The constructing of the Transformer malicious code feature extraction model to perform feature extraction and semantic analysis on the malicious code of the malicious code API vector further includes:

[0026] Use position embedding in Transformer to save the relative or absolute position of the malicious code API function in the sequence:

[0027] PE (pos,2i)=sin(pos / 10000 2i / d ) (14)

[0028] PE (pos,2i+1) =cos(pos / 10000 2i / d ) (15)

[0029] Among them, pos represents the position of each word in the API sequence, d is the dimension of the position vector, which is equal to the dimension of the word encoding vector, i represents the i-th dimension of the position vector, PE represents the position encoding of the current word, and the d-dimensional vector of the pos-th position is obtained according to the above formula.

[0030] The word embedding matrix obtained after the word embedding layer and the position embedding layer is input into the multi-head attention mechanism layer composed of the self-attention mechanism; first, the query, key, and value are obtained through linear transformation, and then the input tensor is reshaped through the separate_head function so that each attention head can process the information in the sequence in parallel, and the attention mechanism of each head is calculated, including calculating the attention score, scaling the attention score, obtaining the attention weight through the SoftMax function, and weighted summing the values ​​through the weights. Finally, the results of each head are summed through linear transformation through Concat+Linear;

[0031] The vector matrix obtained after word embedding of the input sequence is linearly transformed to obtain three sets of vector sequences;

[0032] Q i =W q X i (16)

[0033] K i =W k X i (17)

[0034] V i =W v X i (18)

[0035] Among them, W k , W q , W v is the training weight of the model, which is learned and continuously updated during the training process. i Represents the i-th word vector in the matrix;

[0036] Calculate global weights;

[0037]

[0038] Use Q1 in X1 and perform calculations with all other K. Indicates the length of the key; finally, multiply it with the matrix V to get the weight sum expression;

[0039]

[0040] head i =Attention(QW i Q ,KW i K ,VW i W )(i=1,...,h) (22)

[0041] Z=MultiHead(Q,K,V)=Concat(head1,...,head n )W O (twenty three)

[0042] Among them, Q, K, and V are the query matrix, key matrix, and value matrix of the multi-head attention mechanism, which are equivalent to the fused word vector matrix obtained from the context. k The dimension d of the key k =d v =d' / h; is the transformation matrix, which projects Q, K, and V into h different dimensions. The weight matrix representing multi-head attention;

[0043] Construct residual connections and layer normalization layers:

[0044] L = LayerNorm(X i +Z) (24)

[0045] Where: LayerNorm represents the normalization function, X i Represents the i-th word vector in the word vector matrix;

[0046] The output L of the residual connection and layer normalization is linearly transformed twice using a feedforward neural network and activated using the Relu activation function;

[0047] FFN(L)=max(0,LW1+b1)W2+b2 (25)

[0048] Where: W1, W2 represent the weight matrices of two linear changes, b1, b2 represent the bias matrices;

[0049] The results of the linear transformation and the first layer normalization are subjected to a residual connection and layer normalization, and the result is used as the final output of the Transformer encoder:

[0050] T = LayerNorm(L+FFN(L)) (26)

[0051] Where: T∈R m×d′ .

[0052] The adding an average pooling layer and a fully connected layer to the Transformer malicious code feature extraction model to construct a malicious code detection model further includes:

[0053] Use the average pooling layer to average pool the output information of the Transformer block, calculate the average value of all dimensions, and replace the value of each dimension with the average value;

[0054] Use the Dense fully connected layer to connect each node of the input data to all nodes to form a fully connected network; add the nonlinear activation function ReLU to enable the neural network to learn nonlinear relationships; add the activation function SoftMax to generate the probability distribution of each category, and make the final classification prediction to complete the construction of the malicious code detection model.

[0055] After the malicious code detection model is constructed, the following steps are also included:

[0056] The malicious code detection model is fine-tuned using a model optimization algorithm.

[0057] The present invention also discloses an API semantic vectorization Transformer malicious code detection system using the aforementioned API semantic vectorization Transformer malicious code detection method, comprising:

[0058] A malicious code data extraction module is used to obtain a malicious code data set based on a PE file, extract an API sequence and an MD5 value of the malicious code in the malicious code data set, and determine a malicious code family label according to the MD5 value;

[0059] A negative sampling optimization module, used to build a FastText word embedding model based on negative sampling optimization, and generate a word vector for the API sequence of the malicious code based on the FastText word embedding model;

[0060] Oversampling module, used to oversample malicious code API vectors using the BorederLine-1 SMOTE algorithm and balance data by synthesizing minority class samples at the class boundary;

[0061] A malicious code feature extraction module, used to construct a Transformer malicious code feature extraction model, and perform feature extraction and semantic analysis on the malicious code of the malicious code API vector;

[0062] The malicious code classification detection module is used to add an average pooling layer and a fully connected layer to the Transformer malicious code feature extraction model, build a malicious code detection model, and implement malicious code classification detection.

[0063] Accordingly, the present application also discloses a terminal, including a processor and a storage medium;

[0064] The storage medium is used to store instructions;

[0065] The processor is used to operate according to the instructions to execute the steps of the Transformer malicious code detection method based on the aforementioned API semantic vectorization.

[0066] Correspondingly, the present application also discloses a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the aforementioned API semantic vectorization Transformer malicious code detection method are implemented.

[0067] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides an API semantic vectorization Transformer malicious code detection method and system, which conducts in-depth analysis of the vectorized semantics of malicious code and captures the relationship between low-frequency and similar API functions. First, the FastText word embedding method based on negative sampling optimization is used to generate word vectors, thereby improving the vector representation efficiency of the malicious code data set and the accuracy of low-frequency API vector mapping. Secondly, the Borderline-1SMOTE algorithm is used to increase the number of minority class samples near the decision boundary to enhance the model's learning ability in this area, achieve balanced processing of sample categories, and thus improve the recognition accuracy of the malicious code detection model for rare malicious code families. At the same time, the Transformer model is introduced, and its powerful global context modeling capability is used to perform semantic analysis and feature extraction on the entire malicious code API sequence. The present invention performs feature extraction and semantic analysis on the contextual relationship of the malicious code API sequence, enhances the ability to distinguish similar API sequences, and at the same time improves the ability to understand global information, and can better understand the overall structure and behavior of the malicious code, thereby significantly improving the recognition ability and detection performance of complex and rare malicious code. In addition, the model can more effectively deal with variants and new forms of malicious code, bringing new solutions to the field of network security. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1It is a block diagram of the FastText-based API semantic vectorization Transformer malicious code detection method in the present invention.

[0069] Figure 2 It is a scatter plot of API word vectors based on oversampling of BorderLine-1 SMOTE algorithm in the present invention.

[0070] Figure 3 It is a structural diagram of the Transformer encoder in the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0072] The embodiments described in this application are only some embodiments of the present invention, not all embodiments. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the protection scope of the present invention.

[0073] In view of the shortcomings of the prior art, the present invention proposes an API semantic vectorization Transformer malicious code detection method and system, which uses the FastText word embedding method based on negative sampling optimization to generate word vectors, thereby improving the vector representation efficiency of the malicious code data set and the accuracy of low-frequency API vector mapping. The Borderline-1SMOTE algorithm is used to increase the number of minority class samples near the decision boundary to improve the model's learning ability in this area, achieve balanced processing of sample categories, and thereby improve the recognition accuracy of the malicious code detection model for rare malicious code families. At the same time, the Transformer model is introduced to use its powerful global context modeling capabilities to perform semantic analysis and feature extraction on the entire malicious code API sequence.

[0074] See also Figure 1 As shown, the API semantic vectorization Transformer malicious code detection method disclosed in the present invention includes the following steps:

[0075] Step 1: Acquire a malicious code data set based on PE files, extract the API sequence and MD5 value of the malicious code in the malicious code data set, and determine the malicious code family label according to the MD5 value.

[0076] In a specific embodiment, the step 1 further comprises:

[0077] Step 1.1: Download a malicious code benchmark dataset based on the PE file format from a malicious code related website, and extract the API call sequence and MD5 features of the malicious code in the malicious code benchmark dataset based on the PE format; the PE file of each malicious code sample in the dataset contains features such as SHA256, MD5, tags, header files, release time, API functions, etc. Use python code to extract the MD5 value and API function in the PE header file of each malicious code sample.

[0078] Step 1.2: Distinguish the malicious code family to which each malicious code sample belongs; input the unique MD5 value corresponding to the malicious code API function into the virus scanning website, determine the malicious code family through the virus scanning engine on the virus scanning website, and save the obtained malicious code and malicious code family labels as a .csv file.

[0079] Step 2: Build a FastText word embedding model based on negative sampling optimization, and generate a word vector for the API sequence of the malicious code based on the FastText word embedding model.

[0080] The FastText word embedding model generates word vectors such as Figure 1 The step 2 further comprises:

[0081] Step 2.1: Segment the API sequence of the malicious code and generate a dictionary; use the jieba word segmentation tool to segment the data set, remove irrelevant characters before and after, generate a word list, and then use the tokenizer tool to convert the segmented text data into discrete tokens.

[0082] Step 2.2: Build the input layer of the FastText word embedding model based on negative sampling optimization to obtain the embedding vectors of the n-gram subwords and special subwords of each API function.

[0083] The FastText word embedding model introduces the concept of subword n-gram, and uses character-level n-gram information to capture the sequential relationship between characters, so as to better understand the more subtle semantic relationship within the word. Each API function in the input sequence is processed as an n-gram subword, and each center word is represented as a collection of subwords. "<" and ">" symbols are added to the first position of the word to separate the words, and also to better distinguish between prefix subwords and suffix subwords. Extract n-gram subword information for an API function, and treat the entire word as a special subword, and represent all n-gram subwords and special subwords with embedding vectors.

[0084] Step 2.3: Construct the hidden layer of the FastText word embedding model based on negative sampling optimization, and perform weighted average of the embedding vectors of all subwords.

[0085] Take the average of the embedding vector of each word's n-gram subword and the embedding vector of the special subword obtained in step 2.2. Assume that an API function has only three words W1, W2, and W3. The embedding vectors of these three words are represented as w1, w2, and w3. When the value of n in n-gram is 2, the embedding vector of the n-gram subword is represented as w 12 、w 23 , then the hidden layer vector h of this API function is expressed as:

[0086]

[0087] In the hidden layer, all subword embedding vectors are weighted averaged.

[0088]

[0089] Where W represents the weight matrix, c represents the number of subwords, and x i Represents the embedding vector of the i-th subword, and calculates each node of the output layer in is the j-th column of the matrix W′.

[0090] Step 2.4: Construct the output layer of the FastText word embedding model based on negative sampling optimization and output the malicious code API vector.

[0091] Use softmax to average the hidden layer vector obtained in the hidden layer, multiply it by the inverse transformation matrix W′, and obtain the malicious code API vector h′ whose length is equal to the number of categories.

[0092]

[0093] Then use Negative Sampling to save computing time and improve the mapping accuracy of low-frequency API vectors. The negative sampling loss function is:

[0094]

[0095] where u o represents the embedding vector of the context word, uk is the embedding vector of negative sampling, and v crepresents the embedding vector of the central word, and σ represents the sigmoid function. The first term of the loss function represents the prediction of positive samples, and the second term represents the prediction of negative samples. The smaller the loss function, the better the performance of the model.

[0096] Step 3: Use the BorederLine-1 SMOTE algorithm to oversample the malicious code API vectors and balance the data by synthesizing minority class samples at the boundary of the category.

[0097] In view of the imbalanced distribution of individual categories in the data set, the BorderLine-1 SMOTE algorithm is used to balance the data set by synthesizing minority class samples at the category boundary to balance the data set and improve the distribution of sample categories. After the BorderLine-1 SMOTE algorithm oversamples the malicious code API vector, the API vector scatter plot is as follows: Figure 2 shown.

[0098] Step 4: Construct a Transformer malicious code feature extraction model to perform feature extraction and semantic analysis on the malicious code of the malicious code API vector.

[0099] In the malicious code feature extraction model, the Transformer model is used as the main feature extraction model. Transformer is a model composed entirely of attention mechanisms, mainly including multi-head attention mechanisms, residual networks, and layer normalization. The malicious code API vector processed in step 3 is fed into the Transformer model to extract the malicious code features. The Transformer encoder structure is shown in the figure below. Figure 3 shown.

[0100] Step 4.1: Calculate the position code;

[0101] Use position embedding in Transformer to save the relative or absolute position of the malicious code API function in the sequence:

[0102] PE (pos,2i) =sin(pos / 10000 2i / d ) (31)

[0103] PE (pos,2i+1) =cos(pos / 10000 2i / d ) (32)

[0104] Among them, pos represents the position of each word in the API sequence, d is the dimension of the position vector, which is equal to the dimension of the word encoding vector, i represents the i-th dimension of the position vector, PE represents the position encoding of the current word, and the d-dimensional vector of the pos-th position is obtained according to the above formula.

[0105] Step 4.2: Build a multi-head attention mechanism layer:

[0106] The word embedding matrix obtained after the word embedding layer and the position embedding layer is input into the multi-head attention mechanism layer composed of the self-attention mechanism. First, the query, key, and value are obtained through linear transformation, and then the input tensor is reshaped through the separate_head function so that each attention head can process the information in the sequence in parallel. Then the attention mechanism of each head is calculated, including calculating the attention score, scaling the attention score, obtaining the attention weight through the SoftMax function, and weighted summing the values ​​through the weights. Finally, the results of each head are summed through linear transformation through Concat+Linear.

[0107] Specifically, the vector matrix obtained after word embedding processing of the input sequence is linearly transformed to obtain three sets of vector sequences;

[0108] Q i =W q X i (33)

[0109] K i =W k X i (34)

[0110] V i =W v X i (35)

[0111] Among them, W k , W q , W v is the training weight of the model, which is learned and continuously updated during the training process. i Represents the i-th word vector in the matrix.

[0112] Calculate global weights;

[0113]

[0114] Use Q1 in X1 and perform calculations with all other K. Indicates the length of the key. Finally, multiply it with the matrix V to get the weight sum expression.

[0115]

[0116] head i =Attention(QW i Q ,KWi K ,VW i W )(i=1,…,h) (39)

[0117] Z=MultiHead(Q,K,V)=Concat(head1,...,head n )W O (40)

[0118] Among them, Q, K, and V are the query matrix, key matrix, and value matrix of the multi-head attention mechanism, which are equivalent to the fused word vector matrix obtained from the context. k The dimension d of the key k =d v =d' / h. is the transformation matrix, which projects Q, K, and V into h different dimensions. Represents the weight matrix of multi-head attention.

[0119] Step 4.3: Construct residual connection and layer normalization layer:

[0120] L = LayerNorm(X i +Z) (41)

[0121] Where: LayerNorm represents the normalization function, X i Represents the i-th word vector in the word vector matrix.

[0122] The output L of the residual connection and layer normalization process is linearly transformed twice using a feedforward neural network and activated using the ReLU activation function.

[0123] FFN(L)=max(0,LW1+b1)W2+b2 (42)

[0124] Where: W1, W2 represent the weight matrices of two linear changes, b1, b2 represent the bias matrices.

[0125] Step 4.4: Transformer encoder model output:

[0126] Finally, the results of the linear transformation and the first layer normalization are subjected to a residual connection and layer normalization again, and the result is used as the final output of the Transformer encoder.

[0127] T = LayerNorm(L+FFN(L)) (43)

[0128] Where: T∈R m×d′ .

[0129] Step 5: Add an average pooling layer and a fully connected layer to the Transformer malicious code feature extraction model to build a malicious code detection model to achieve malicious code classification detection.

[0130] The average pooling layer is used to average pool the output information of the Transformer block, calculate the average value of all its dimensions, and replace the value of each dimension with the average value. This can help reduce the length of the sequence, extract key information, and reduce the number of parameters in the model. At the same time, it helps to capture the overall characteristics of the sequence without paying attention to the information at a specific position, thereby improving the generalization ability of the model to a certain extent.

[0131]

[0132] Among them, y represents the output after pooling, n represents the number of elements in the pooling window, that is, the size of the pooling window, and x i Represents the value of each element in the pooling window.

[0133] Finally, the Dense fully connected layer is used to connect each node of the input data to all nodes to form a fully connected network. The nonlinear activation function ReLU is added to enable the neural network to learn nonlinear relationships and better fit complex data. The probability distribution of each category is generated by adding the activation function SoftMax, and the final classification prediction is performed to complete the construction of the malicious code detection model.

[0134] y=f(Wx+b) (45)

[0135] Among them, y represents the output of the fully connected layer, W represents the weight matrix, x represents the input vector, and b represents the bias vector.

[0136] In a further preferred embodiment, the method further comprises:

[0137] Step 6: Use a model optimization algorithm to fine-tune the malicious code detection model.

[0138] In order to ensure that the model is more accurate in detecting malicious code and to reduce the occurrence of overfitting, the present invention uses the k-fold cross-validation method to fine-tune the model and divides the data set into a training set and a test set in a 9:1 ratio.

[0139]

[0140] Among them, CV score represents the average score of k-fold cross validation, k represents the number of folds, score i Represents the model evaluation score on the i-th fold.

[0141] In a further preferred embodiment, the method further comprises:

[0142] Step 7: Perform performance evaluation on the malicious code detection model.

[0143] The accuracy, loss function, recall rate and F1 value are used as evaluation indicators of the malicious code detection method to evaluate the performance of the model. The malicious code detection model uses 10-fold cross validation and the number of iterations is 20. After all the iterations of the model are completed, the accuracy rate can reach 99.66%, the loss function value can reach 0.0066, the recall rate can reach 99.58%, and the F1 value can reach 99.58%.

[0144] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides an API semantic vectorization Transformer malicious code detection method and system, which conducts in-depth analysis of the vectorized semantics of malicious code and captures the relationship between low-frequency and similar API functions. First, the FastText word embedding method based on negative sampling optimization is used to generate word vectors, thereby improving the vector representation efficiency of the malicious code data set and the accuracy of low-frequency API vector mapping. Secondly, the Borderline-1SMOTE algorithm is used to increase the number of minority class samples near the decision boundary to enhance the model's learning ability in this area, achieve balanced processing of sample categories, and thus improve the recognition accuracy of the malicious code detection model for rare malicious code families. At the same time, the Transformer model is introduced, and its powerful global context modeling capability is used to perform semantic analysis and feature extraction on the entire malicious code API sequence. The present invention performs feature extraction and semantic analysis on the contextual relationship of the malicious code API sequence, enhances the ability to distinguish similar API sequences, and at the same time improves the ability to understand global information, and can better understand the overall structure and behavior of the malicious code, thereby significantly improving the recognition ability and detection performance of complex and rare malicious code. In addition, the model can more effectively deal with variants and new forms of malicious code, bringing new solutions to the field of network security.

[0145] The present invention may be a system, a method and / or a computer program product. The present invention also discloses an API semantic vectorization Transformer malicious code detection system based on the aforementioned API semantic vectorization Transformer malicious code detection method, comprising:

[0146] A malicious code data extraction module is used to obtain a malicious code data set based on a PE file, extract an API sequence and an MD5 value of the malicious code in the malicious code data set, and determine a malicious code family label according to the MD5 value;

[0147] A negative sampling optimization module, used to build a FastText word embedding model based on negative sampling optimization, and generate a word vector for the API sequence of the malicious code based on the FastText word embedding model;

[0148] Oversampling module, used to oversample malicious code API vectors using the BorederLine-1 SMOTE algorithm and balance data by synthesizing minority class samples at the class boundary;

[0149] A malicious code feature extraction module, used to construct a Transformer malicious code feature extraction model, and perform feature extraction and semantic analysis on the malicious code of the malicious code API vector;

[0150] The malicious code classification detection module is used to add an average pooling layer and a fully connected layer to the Transformer malicious code feature extraction model, build a malicious code detection model, and implement malicious code classification detection.

[0151] Based on the spirit of the present invention, those skilled in the art can easily think of a computer program product based on the aforementioned API semantic vectorization Transformer malicious code detection method. The computer program product may include a computer-readable storage medium, which carries computer-readable program instructions for enabling a processor to implement various aspects of the present disclosure. That is, the present application also includes a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the aforementioned API semantic vectorization Transformer malicious code detection method.

[0152] Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device. Computer readable storage medium can be, for example, - but not limited to - electrical storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, for example, punch card or groove protrusion structure with instructions stored thereon and any suitable combination of the above. Computer readable storage medium used here is not interpreted as instantaneous signal itself, such as radio wave or other free propagating electromagnetic wave, electromagnetic wave propagated by waveguide or other transmission medium (for example, light pulse by optical fiber cable) or electrical signal transmitted by wire.

[0153] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0154] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages-such as Smalltalk, C++, etc., and conventional procedural programming languages-such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network-including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An API semantic vectorization Transformer malicious code detection method, characterized in that: The following steps are involved: Step 1: Obtain a malicious code data set based on a PE file, extract the API sequence and MD5 value of the malicious code in the malicious code data set, and determine the malicious code family label according to the MD5 value; Step 2: construct a FastText word embedding model based on negative sampling optimization, and generate a word vector for the API sequence of the malicious code based on the FastText word embedding model; Step 3: Use the BorederLine-1 SMOTE algorithm to oversample the malicious code API vectors and balance the data by synthesizing minority class samples at the boundary of the class. Step 4: construct a Transformer malicious code feature extraction model to perform feature extraction and semantic analysis on the malicious code of the malicious code API vector; Step 5: Add an average pooling layer and a fully connected layer to the Transformer malicious code feature extraction model to build a malicious code detection model to achieve malicious code classification detection.

2. The API semantic vectorization Transformer malicious code detection method according to claim 1 is characterized in that: The extracting the API sequence and MD5 value of the malicious code in the malicious code data set, and determining the malicious code family label according to the MD5 value, further includes: Step 1.1: Download a malicious code benchmark dataset based on the PE file format, and extract the API call sequence and MD5 features of the malicious code in the malicious code benchmark dataset based on the PE format; the PE file of each malicious code sample in the dataset contains SHA256, MD5, tags, header files, release time, and API functions. Use Python code to extract the MD5 value and API function in the PE header file of each malicious code sample; Step 1.2: Distinguish the malicious code family to which each malicious code sample belongs; input the unique MD5 value corresponding to the malicious code API function into the virus scanning website, determine the malicious code family through the virus scanning engine of the virus scanning website, and save the obtained malicious code and malicious code family labels as a .csv file.

3. The API semantic vectorization Transformer malicious code detection method according to claim 2 is characterized in that: The constructing of a FastText word embedding model based on negative sampling optimization, and generating a word vector from the API sequence of the malicious code based on the FastText word embedding model, further includes: Step 2.1: Segment the API sequence of the malicious code and generate a dictionary; use the jieba word segmentation tool to segment the data set, remove irrelevant characters before and after, segment according to the API sequence and API call function, generate a word list, and then use the tokenizer tool to convert the segmented text data into discrete tokens; Step 2.2: Build the input layer of the FastText word embedding model based on negative sampling optimization to obtain the embedding vectors of the n-gram subwords and special subwords of each API function; Step 2.3: Construct the hidden layer of the FastText word embedding model based on negative sampling optimization, and perform weighted average of the embedding vectors of all subwords; Step 2.4: Construct the output layer of the FastText word embedding model based on negative sampling optimization and output the malicious code API vector.

4. The API semantic vectorization Transformer malicious code detection method according to claim 3 is characterized in that: The construction of the FastText word embedding model input layer based on negative sampling optimization, obtaining the embedding vectors of the n-gram subwords and special subwords of each API function, further includes: Character-level n-gram information is used to capture the sequential relationship between characters. Each API function in the input sequence is processed into n-gram subwords, and each central word is represented as a collection of subwords. "<" and "">" symbols are added to the first position of a word to separate words and distinguish prefix subwords from suffix subwords. The n-gram subword information of an API function is extracted, and the word as a whole is regarded as a special subword. All n-gram subwords and special subwords are represented by embedding vectors.

5. The API semantic vectorization Transformer malicious code detection method according to claim 4 is characterized in that: The constructing of the Transformer malicious code feature extraction model to perform feature extraction and semantic analysis on the malicious code of the malicious code API vector further includes: Use position embedding in Transformer to save the relative or absolute position of the malicious code API function in the sequence: ON (pos,2i) =sin(pos / 10000 2i / d ) (1) ON (pos,2i+1) =cos(pos / 10000 2i / d ) (2) Among them, pos represents the position of each word in the API sequence, d is the dimension of the position vector, which is equal to the dimension of the word encoding vector, i represents the i-th dimension of the position vector, PE represents the position encoding of the current word, and the d-dimensional vector of the pos-th position is obtained according to the above formula. The word embedding matrix obtained after the word embedding layer and the position embedding layer is input into the multi-head attention mechanism layer composed of the self-attention mechanism; first, the query, key, and value are obtained through linear transformation, and then the input tensor is reshaped through the separate_head function so that each attention head can process the information in the sequence in parallel, and the attention mechanism of each head is calculated, including calculating the attention score, scaling the attention score, obtaining the attention weight through the SoftMax function, and weighted summing the values ​​through the weights. Finally, the results of each head are summed through linear transformation through Concat+Linear; The vectors obtained after word embedding of the input sequence are placed into a vector matrix X, and three sets of vector sequences are obtained after linear transformation; Q i =W q X i (3) K i =W k X i (4) V i =W v X i (5) Among them, W k , W q , W v is the training weight of the model, which is learned and continuously updated during the training process. i Represents the i-th word vector in the matrix; Calculate global weights; Use Q1 in X1 and perform calculations with all other K. Indicates the length of the key; finally, multiply it with the matrix V to get the weight sum expression; head i =Attention(QW i Q ,KW i K ,VW i W )(i=1,…,h) (9) Z=MultiHead(Q,K,V)=Concat(head1,...,head n )W O (10) Among them, Q, K, and V are the query matrix, key matrix, and value matrix of the multi-head attention mechanism, which are equivalent to the fused word vector matrix obtained from the context. k The dimension d representing the key k =d v =d' / h; is the transformation matrix, which projects Q, K, and V into h different dimensions. The weight matrix representing multi-head attention; Construct residual connections and layer normalization layers: L=LayerNorm(X i +Z) (11) Where: LayerNorm represents the normalization function, X i Represents the i-th word vector in the word vector matrix; The output L of the residual connection and layer normalization is linearly transformed twice using a feedforward neural network and activated using the Relu activation function; FFN(L)=max(0,LW1+b1)W2+b2 (12) Where: W1, W2 represent the weight matrices of two linear changes, b1, b2 represent the bias matrices; The results of the linear transformation and the first layer normalization are subjected to a residual connection and layer normalization, and the result is used as the final output of the Transformer encoder: T = LayerNorm(L+FFN(L)) (13) Where: T∈R m×d' .

6. The API semantic vectorization Transformer malicious code detection system according to claim 5, characterized in that: The adding an average pooling layer and a fully connected layer to the Transformer malicious code feature extraction model to construct a malicious code detection model further includes: Use the average pooling layer to average pool the output information of the Transformer block, calculate the average value of all dimensions, and replace the value of each dimension with the average value; Use the Dense fully connected layer to connect each node of the input data to all nodes to form a fully connected network; add the nonlinear activation function ReLU to enable the neural network to learn nonlinear relationships; add the activation function SoftMax to generate the probability distribution of each category, and make the final classification prediction to complete the construction of the malicious code detection model.

7. The API semantic vectorization Transformer malicious code detection system according to claim 6, characterized in that: After the malicious code detection model is constructed, the following steps are also included: The malicious code detection model is fine-tuned using a model optimization algorithm.

8. An API semantic vectorization Transformer malicious code detection system, characterized in that: include: A malicious code data extraction module is used to obtain a malicious code data set based on a PE file, extract an API sequence and an MD5 value of the malicious code in the malicious code data set, and determine a malicious code family label according to the MD5 value; A negative sampling optimization module, used to build a FastText word embedding model based on negative sampling optimization, and generate a word vector for the API sequence of the malicious code based on the FastText word embedding model; Oversampling module, used to oversample malicious code API vectors using the BorederLine-1 SMOTE algorithm and balance data by synthesizing minority class samples at the class boundary; A malicious code feature extraction module, used to construct a Transformer malicious code feature extraction model, and perform feature extraction and semantic analysis on the malicious code of the malicious code API vector; The malicious code classification detection module is used to add an average pooling layer and a fully connected layer to the Transformer malicious code feature extraction model, build a malicious code detection model, and implement malicious code classification detection.

9. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the API semantic vectorization Transformer malicious code detection method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the API semantic vectorization Transformer malicious code detection method described in any one of claims 1 to 7 are implemented.

Images