Malicious code classification system based on FPN-Transform model
By combining the FPN-Transformer model with a feature pyramid network and hash random compression, feature extraction and classification are optimized, solving the problems of dependence on high-quality data and model collapse in existing malware classification methods. This achieves higher accuracy and lower resource consumption, and is adaptable to different types of malware classification.
Patent Information
- Application Number
- CN202511228206.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2025-12-19
AI Technical Summary
Existing malware classification methods based on deep learning networks rely on high-quality data and are susceptible to neural network model collapse or overfitting, resulting in decreased model generalization ability and practicality. In particular, they consume a lot of resources and have insufficient accuracy when dealing with large-scale or low-quality datasets.
A malware classification system based on the FPN-Transformer model is adopted. By improving the data preprocessing and feature extraction mechanism, combining the Feature Pyramid Network (FPN) and the Transformer model, the feature representation and classification efficiency are optimized, the dependence on high-quality data is reduced, and the accuracy and generalization ability of the model are improved by hash random compression, multi-level regression feature extraction and autocorrelation matrix classifier.
It improves the accuracy and generalization ability of malicious code classification, reduces the dependence on high-quality data, reduces the consumption of computing resources, and enhances the stability and adaptability of the model, especially when dealing with large-scale or low-quality datasets.
Smart Images
Figure CN121167720A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of network security, specifically relating to a malicious code classification system based on the FPN-Transformer model. Background Technology
[0002] Malicious code families are typically classified by monitoring network activity and analyzing the behavioral characteristics of malware. These classification methods serve a specific purpose: analyzing the activity patterns and attack strategies of malware in different network environments. Many malware families exhibit similar attack characteristics, but their behavior is not entirely identical; rather, they possess variant features. These features may represent specific attack intentions or malicious activities, such as data theft, system disruption, or other security threats. Therefore, the accurate identification and classification of these malware family characteristics is of significant practical importance for improving the effectiveness of network security protection measures.
[0003] The sample size of malware is enormous and growing explosively. Early rule-based and signature-based classification methods, while making some progress, suffer from slow pattern updates and high false positive rates. With the development of machine learning technology, especially the introduction of deep learning models, malware detection and classification techniques have made significant progress. However, traditional deep learning methods often rely on large amounts of high-quality labeled data when processing malware and are susceptible to "neural network model collapse" or "overfitting," leading to decreased generalization ability and practicality of classification models in real-world applications. Existing classification methods often face problems such as insufficient feature extraction and high resource consumption when processing large-scale datasets when dealing with complex and ever-changing malware families. Summary of the Invention
[0004] The technical problem of this invention is that existing malware classification methods based on deep learning networks rely on high-quality data and are susceptible to neural network model collapse or overfitting, resulting in a decline in model generalization ability and practicality. In particular, they face problems of high resource consumption and insufficient accuracy when dealing with large-scale or low-quality datasets.
[0005] The purpose of this invention is to address the above-mentioned problems by providing a malware classification system based on the FPN-Transformer model. This system reduces the dependence on high-quality data by improving data preprocessing and feature extraction mechanisms, and optimizes feature representation and classification efficiency by combining Feature Pyramid Networks (FPN) with the Transformer model, thereby improving the accuracy and generalization ability of the model.
[0006] To achieve the above objectives, in a first aspect, the present invention provides a malicious code classification system based on the FPN-Transformer model, comprising the following modules: Data input module: Receives code files or code assemblies as input; Data preprocessing module: preprocesses the code data obtained from the data input module, including feature extraction, data cleaning, standardization / normalization and feature selection, to ensure the quality and consistency of the input data, and outputs the preprocessed feature data to the Hash random compression module; Hash random compression module: Uses hash algorithm to randomly map and compress preprocessed feature data into hash space; Feature selection module: Based on the set threshold, the feature data mapped and compressed by the hash algorithm is filtered, i.e., feature selection is performed to retain features that make a significant contribution to the classification of malicious code families and generate a compressed feature set; FPN-Transformer Model Module: Utilizes the FPN-Transformer model to perform in-depth analysis and feature extraction on the feature data output by the feature selection module; Multi-level regression feature extraction model module: The multi-level regression feature extraction model is used to extract multi-scale feature information from the feature data output by the FPN-Transformer model, generate channel attention features and spatial attention features and fuse them, use a linear layer to perform linear transformation on the fused features, and input the linearly transformed features into the regression layer to generate the autocorrelation matrix. Classification result output module: Based on the autocorrelation matrix obtained from the multi-level regression feature extraction model, determine whether there is malicious code in the code file, obtain the malicious code family to which the malicious code belongs, and output the final classification result.
[0007] Preferably, the FPN-Transformer model specifically includes multiple average pooling layers, M encoders, and M decoders, where M is the number of encoders. The encoder includes an input embedding layer, a multi-head attention unit, a normalization layer, a feedforward neural network, and a normalization layer connected in sequence. The decoder has the same structure as the encoder. The multi-head attention unit with self-attention mechanism and the feedforward neural network are used for feature transformation and enhancement.
[0008] Preferably, the multi-layer regression feature extraction model specifically includes a parallel channel attention module and a spatial attention module, a linear layer and a regression layer, and an autocorrelation matrix classifier. The channel attention module includes an input embedding layer, parallel average pooling layers and max pooling layers, and a shared multilayer perceptron. The spatial attention module includes an input embedding layer, parallel average pooling layers and max pooling layers, and a convolutional layer. The channel attention module, spatial attention module, linear layer, and regression layer are combined to capture multi-scale information of malicious code features. The shared multilayer perceptron of the channel attention module is combined with the regression layer to enhance the richness and robustness of feature representation. The autocorrelation matrix classifier uses the autocorrelation matrix to evaluate and strengthen the relationship between classification features, and optimizes feature combination through autocorrelation to improve classification accuracy and robustness.
[0009] Secondly, this invention provides a malicious code classification method based on the FPN-Transformer model, comprising the following steps: Step 1: Data preprocessing. Use natural language processing models to extract preliminary feature data from the code files or code assemblies to be classified, and capture the semantic and structural information of the code. Step 2: Perform hash random compression on the feature data obtained in Step 1 to reduce the dimensionality of the feature data and reduce noise; Step 3: Use the FPN-Transformer model to perform in-depth analysis and feature extraction on the compressed feature data; Step 4: The feature data obtained in Step 3 is further fused and classified using a multi-level regression feature extraction model; Step 5: Based on the output of the multi-level regression feature extraction model, determine whether there is malicious code in the code file, and determine the malicious code family to which the malicious code belongs.
[0010] Preferably, the code in the code file or code assembly to be classified is in binary or script format.
[0011] Preferably, step 1 specifically includes the following sub-steps: Step 1.1: Use the CodeBERT model to extract feature data from the code file, including Abstract Syntax Tree (AST), Node, Code, and Entropy; Step 1.2: Remove noise and irrelevant information from the feature data, and delete null or outlier values; Step 1.3: Standardize or normalize the feature data to ensure comparability between different features; the formula used for normalization is as follows, where... The mean, Standard deviation: ; In the formula The mean, Standard deviation, , These are the feature data before and after normalization, respectively. Step 1.4: Select feature data that is useful for the classification task and remove redundant feature data.
[0012] Preferably, step 2 specifically includes the following sub-steps: Step 2.1: Use a hash function to map the preprocessed feature values to the hash function space; Step 2.2: Select important features based on preset thresholds to reduce the influence of irrelevant features; Preferably, step 3 specifically includes: Step 3.1: Convert the input sample feature data into embedding vectors; Step 3.2: Perform average pooling operations of different magnitudes on the embedded vectors to generate multiple sets of feature mapping data; Step 3.3: Concatenate the multiple sets of feature mapping data obtained in Step 3.2 along the feature dimension to form a feature patch; Step 3.4: Input the feature patch obtained in Step 3.3 into the encoder for processing; Step 3.4.1: Calculate the correlation between input features using a multi-head attention mechanism unit to generate an attention weight matrix; Step 3.4.2: The feedforward neural network performs a nonlinear transformation on the features output by the multi-head attention mechanism unit; Step 3.4.3: Normalize the feature data output by the feedforward neural network; Step 3.5: Input the feature data output by the encoder into the decoder for processing. The data processing process of the decoder is the same as in step 3.4, to obtain multi-level and multi-view feature data.
[0013] Preferably, step 4 specifically includes: 1) Use the channel attention module to extract multi-scale feature information from the feature data output by the FPN-Transformer model to generate channel attention features; 1.1) The input feature representation is converted into an embedding vector through the input embedding layer; 1.2) Use average pooling (AvgPool) and max pooling (MaxPool) operations to generate two sets of feature maps; 1.3) Input the pooled features into a shared multilayer perceptron for feature transformation; 1.4) The features transformed by the shared multilayer perceptron are fused through element-wise addition and element-wise multiplication operations to generate channel attention features; 2) Use the spatial attention module to extract multi-scale feature information from the feature data output by the FPN-Transformer model to generate spatial attention features; 2.1) Use an input embedding layer to convert the input feature representation into an embedding vector; 2.2) Two sets of feature maps are generated using average pooling (AvgPool) and max pooling (MaxPool) operations; 2.3) Input the pooled features into the convolutional layer Conv for feature convolution operation; 2.4) The convolutional features are fused using element-wise addition and element-wise multiplication operations to generate spatial attention features; 3) Fuse the channel attention features and spatial attention features to generate the final attention feature representation; use a linear layer to perform a linear transformation on the fused features; input the linearly transformed features into a regression layer to generate an autocorrelation matrix.
[0014] Compared with the prior art, the beneficial effects of the present invention include: 1) This invention combines the Feature Pyramid Network (FPN) and the Transformer model to construct the FPN-Transformer model, providing multi-level and multi-perspective feature data for the multi-level regression feature extraction model. Then, the multi-level regression feature extraction model is used to integrate and fuse the features extracted from different levels by the FPN-Transformer model. Compared with existing malware classification methods that use a single classification model, this invention uses two models in succession: first, the FPN-Transformer model is used for input data feature decomposition and mining, and then the multi-level regression feature extraction model is used for attention weight enhancement, feature extraction, and feature fusion. The fused features are input into the autocorrelation matrix classifier to obtain the malware classification. This effectively improves the accuracy and generalization ability of the malware classification method, prevents overfitting of the classification model and avoids the collapse of the classification model, and improves the stability of the classification model.
[0015] 2) This invention utilizes a hash function to select features from the input data of the FPN-Transformer model. The hash function's randomness allows for dynamic mapping and compression of features, enabling the model to better adapt to different types of malware and reducing the risk of overfitting. Traditional feature selection methods, such as those based on statistical features or information gain, are typically fixed. This means that once certain features are selected, the model will rely on them throughout subsequent training. These existing methods often fall short when dealing with dynamic and complex malware. This invention combines a hash-based random compression method with a deep learning model, enabling adaptive learning and selection of key features, thus improving the accuracy and efficiency of malware classification.
[0016] 3) This invention utilizes the FPN-Transformer model for in-depth analysis and feature extraction of malware feature data. Compared to traditional deep learning methods, this parallel decomposition mechanism for obtaining multi-scale features is more flexible and detailed. The Transformer's self-attention mechanism allows the model to focus more on the relationships and contextual information between features when analyzing them. This mechanism is particularly useful in malware classification. Compared to deep learning models like LSTM, the FPN-Transformer module can efficiently capture long-distance dependencies by processing features in parallel and utilizing the self-attention mechanism, without needing to iterate through sequences step by step. This makes it more efficient and effective in handling complex malware classification tasks.
[0017] 4) This invention uses a multi-layer regression feature extraction model to synthesize and fuse multi-level, multi-scale features output by the FPN-Transformer model. This model not only performs simple feature extraction but also extracts multi-scale information from the features output by the FPN-Transformer model and fuses features through channel attention and spatial attention mechanisms. Traditional deep learning models, such as single-layer fully connected networks or single convolutional layers, typically only capture information from a single level or dimension of the input features. This invention's multi-layer regression feature extraction model, through multi-level regression analysis, not only extracts and fuses multi-scale features but also delves into the complex relationships between features through channel attention and spatial attention mechanisms. This deep feature fusion enables the model to better capture the multi-dimensional features of malicious code, thereby achieving better results in classification tasks.
[0018] 5) This invention utilizes an autocorrelation matrix classifier to fuse the feature data output by parallel channel attention modules and spatial attention modules, thereby optimizing the feature structure graph. This facilitates accurate classification of the malware family to which the malware belongs, further enhancing the accuracy of this invention in identifying features of different malware families.
[0019] 6) The model of this invention can adaptively learn feature representations of different data distributions, thereby more accurately identifying and classifying malware families, improving classification accuracy and efficiency, and reducing dependence on high-quality training data, making it more efficient when processing large-scale or low-quality datasets. The method of this invention reduces computational resource consumption and improves the feasibility and effectiveness of the model in practical applications. Attached Figure Description
[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0021] Figure 1 This is a schematic diagram of the data flow of the malicious code classification method in an embodiment of the present invention.
[0022] Figure 2 This is a schematic diagram of the Hash random compression process according to an embodiment of the present invention.
[0023] Figure 3 This is a schematic diagram of the FPN-Transformer model according to an embodiment of the present invention.
[0024] Figure 4 This is a schematic diagram of a multi-level regression feature extraction model according to an embodiment of the present invention.
[0025] Figure 5 This is a comparison chart of the classification performance of the present invention and the Transformer-CNN model. Detailed Implementation
[0026] To improve the accuracy and efficiency of malware family classification, this invention proposes a malware classification system and method based on the FPN-Transformer model. The main implementation principles, specific implementation processes, and corresponding beneficial effects of this invention will be described in detail below with reference to the accompanying drawings.
[0027] The malware classification system based on the FPN-Transformer model in this embodiment includes the following modules: Data input module: Receives code files or code assemblies as input; Data preprocessing module: preprocesses the code data obtained from the data input module, including feature extraction, data cleaning, standardization / normalization and feature selection, to ensure the quality and consistency of the input data, and outputs the preprocessed feature data to the Hash random compression module; Hash random compression module: Uses hash algorithm to randomly map and compress preprocessed feature data into hash space; Feature selection module: Based on the set threshold, the feature data mapped and compressed by the hash algorithm is filtered, i.e., feature selection is performed to retain features that make a significant contribution to the classification of malicious code families and generate a compressed feature set; FPN-Transformer Model Module: Utilizes the FPN-Transformer model to perform in-depth analysis and feature extraction on the feature data output by the feature selection module; Multi-level regression feature extraction model module: The multi-level regression feature extraction model is used to extract multi-scale feature information from the feature data output by the FPN-Transformer model, generate channel attention features and spatial attention features and fuse them, use a linear layer to perform linear transformation on the fused features, and input the linearly transformed features into the regression layer to generate the autocorrelation matrix. Classification result output module: Based on the autocorrelation matrix obtained from the multi-level regression feature extraction model, determine whether there is malicious code in the code file, obtain the malicious code family to which the malicious code belongs, and output the final classification result.
[0028] like Figure 1 As shown, the method of the malware classification system based on the FPN-Transformer model includes: Step 1: Extract the malicious code source file from the open-source dataset PS and perform data preprocessing.
[0029] Step 1.1: Extract the required features from the input data, including Abstract Syntax Tree (AST), Node, Code, Entropy, and other features.
[0030] Step 1.2: Remove noise and irrelevant information from the data, delete null or outlier values, and ensure data quality.
[0031] Step 1.3: Normalize the data to ensure comparability between different features. The normalization formula is: ; In the formula The mean, Standard deviation, , These are the feature data before and after normalization, respectively.
[0032] Step 1.4: Select features that are useful for the classification task and remove redundant features.
[0033] Step 2: Perform hash-based random compression on the feature data obtained in Step 1 to reduce the dimensionality of the feature data and decrease noise, such as... Figure 2 As shown.
[0034] Step 2.1: Use a hash function to map the preprocessed feature values to the hash function space.
[0035] Step 2.1.1: Select a suitable hash function for data preprocessing. The selected hash function should have efficient mapping and low collision characteristics.
[0036] Step 2.1.2: Transfer the feature data obtained in Step 1 Divided into multiple small pieces The hash function selected in step 2.1.1 is used to process each small block. Random mapping and compression are performed, and the hash function is: ; In the formula h Represents a hash function. i is a feature vector. i is a random vector. It is a prime number. The size of the hash space; d Indicates the size of the feature dimension.
[0037] Step 2.2: Select important features based on preset thresholds to reduce the influence of irrelevant features.
[0038] Step 2.2.1: Set the threshold σ based on the feature value distribution and feature importance metric; Step 2.2.2: Retain features below the threshold σ and discard other features. The calculation formula is: ; In the formula This represents the hash feature data obtained through filtering.
[0039] Step 3: Input the filtered features into the FPN-Transformer classification model, such as... Figure 3 As shown, the FPN-Transformer model is used to perform in-depth analysis and feature extraction on the compressed feature data.
[0040] Step 3.1: Convert the sample feature data input to the FPN-Transformer classification model into an embedding vector. The dimension of the sample feature data is N×128, where N is the number of samples and the feature dimension is 128.
[0041] Step 3.2: Perform four average pooling operations of different sizes on the embedded vector to generate four sets of feature maps. The pooling window sizes are 2, 4, 8 and 16, respectively, and the feature dimensions after pooling are N×64, N×32, N×16 and N×8, respectively.
[0042] Step 3.3: Concatenate the four pooled feature maps along the feature dimension to form a large feature patch.
[0043] Step 3.4: Input the feature patches into the encoders for processing. The number of encoders is adjusted based on the model's performance to find the optimal parameters. In this embodiment, six encoders are used. Each encoder contains a multi-head attention mechanism unit, a feedforward neural network, and a normalization layer.
[0044] Step 3.4.1: The multi-head attention mechanism unit calculates the correlation between input features, generates an attention weight matrix, and uses 8 attention heads to capture global information.
[0045] Step 3.4.2: Use a feedforward neural network to perform a nonlinear transformation on the features output by the attention mechanism. The hidden layer dimension of the feedforward neural network used is 512.
[0046] Step 3.4.3: Normalization layer normalizes the output of each layer, and the feature representation dimension after feature generation is N×120.
[0047] Step 3.5: Input the features output by the encoder into the decoder for processing. The decoder structure is similar to the encoder, including a multi-head attention mechanism, a feedforward neural network, and a normalization layer. In this embodiment, six decoders are used, just like the encoder. The data processing procedure for each decoder is the same as that for the encoder in Step 3.4, and the final feature representation after feature reconstruction has a dimension of N×120.
[0048] Step 4: The features reconstructed from the encoder features in Step 3 are further fused and classified using a multi-level regression feature extraction model, such as... Figure 4 As shown.
[0049] Step 4.1: Use the channel attention module to extract multi-scale feature information from the feature data output by the FPN-Transformer model to generate channel attention features.
[0050] Step 4.1.1: In the channel attention module, the input feature representation is converted into an embedding vector suitable for subsequent processing through the input embedding layer.
[0051] Step 4.1.2: Use average pooling (AvgPool) and max pooling (MaxPool) operations to generate two sets of feature maps. The feature dimension after pooling remains N×120N.
[0052] Step 4.1.3: Input the pooled features into a Shared MLP for feature transformation. The Shared MLP transforms the feature dimension to N×60N.
[0053] Step 4.1.4: The transformed features are fused using element-wise addition (Sum) and element-wise multiplication (Multiply) to generate channel attention features.
[0054] Step 4.2: Use the spatial attention module to extract multi-scale feature information from the feature data output by the FPN-Transformer model to generate spatial attention features.
[0055] Step 4.2.1: In the spatial attention module, the input feature representation is converted into an embedding vector using the input embedding layer.
[0056] Step 4.2.2: Use average pooling (AvgPool) and max pooling (MaxPool) operations to generate two sets of feature maps. The feature dimension after pooling remains N×120N.
[0057] Step 4.2.3: Input the pooled features into the convolutional layer Conv and perform feature convolution. The feature dimension after convolution remains N×120.
[0058] Step 4.2.4: The convolutional features are fused using element-wise addition (Sum) and element-wise multiplication (Multiply) operations to generate spatial attention features.
[0059] Step 4.3: Fuse the channel attention features and spatial attention features to generate the final attention feature representation.
[0060] Step 4.4: Use a linear layer to perform a linear transformation on the fused features, with an output dimension of N×120.
[0061] Step 4.5: Input the linearly transformed features into the regression layer to generate an autocorrelation matrix, which is used for the final feature classification. The output dimension of the autocorrelation matrix is N×k, where k is the number of labels for the classification task. The mathematical formula for the autocorrelation matrix classifier is: ; In the formula This represents the initialization parameters in the tag space. This represents a parameter in another dimension of the label space. Represents the Kronecker delta function. Labels used to distinguish different categories and It represents the similarity or angular relationship between two vectors in the label space in a higher-dimensional space.
[0062] Step 5: Output the predicted label category and compare it with the true label category to calculate the loss value.
[0063] In this embodiment, the training of the FPN-Transformer model and the multilevel regression feature extraction model specifically includes: 1) Generate a training sample dataset, which contains samples of actual network traffic or pre-designed malicious code; 2) Label the training sample dataset by adding a category label to each sample image, which indicates which malware family the sample belongs to; 3) The dataset contains 1,000-5,000 samples, and the balance of samples from each family is ensured to improve the generalization ability and accuracy of the model.
[0064] In this embodiment, the malware dataset PS, publicly available on GitHub by White, was used. This dataset contains samples from multiple malware families and is divided into six families: Class0 = Downloader, Class1 = Shellcode Inject, Class2 = Unicorn, Class3 = Powerfun, Class4 = SET, and Class5 = PowerShellEmpire. The runtime environment for this embodiment consisted of an Intel(R) Xeon(R) Gold 6330 CPU @ 2.00GHz, a GeForce RTX 3090 GPU, 500GB of RAM, and Python version 3.7. PyTorch and Sklearn were used to implement and test the method described in this paper.
[0065] In this embodiment, the model iteration count is uniformly set to 100 times, the feature dimension of the input data is 512, the hidden layer dimension is 64, the number of heads in the multi-head attention mechanism of the FPN-Transformer model is 8, the number of encoder layers is 1, the number of decoder layers is 2, the optimization algorithm is the Adam algorithm, and the learning rate is set to 0.0001.
[0066] Experimental results show that the method of this invention achieves a classification accuracy of 97.8% on White's PS dataset. The precision, recall, and F1 score also reach high levels, effectively targeting malware families such as Downloader and ShellcodeInject. The classification accuracy of Unicorn, Powerfun, SET, and PowerShell Empire is shown in Table 1.
[0067] In addition, this invention was compared with the latest malware classification model, Transformer-CNN, and the comparison results are as follows: Figure 5As shown in Figure 5(a), this is a scatter plot of the data distribution of different categories in the original dataset. Although the boundaries between the data points of different categories are relatively clear, there is still a certain degree of mixing. Figure 5 Figures 5(b) and 5(d) show the clustering results of the Transformer-CNN model and the method of this invention. The figure shows the effect when high-dimensional data is reduced to a two-dimensional plane. Figure 5 Figures 5(c) and 5(e) compare the fitting between the predicted values of the Transformer-CNN model and the method of the present invention and the true labels using line graphs. The results show that the predicted line graph of the method of the present invention is closer to the true label line graph and shows a smaller deviation, which confirms the superiority of the method of the present invention in terms of accuracy and generalization ability.
[0068] Table 1. Classification accuracy of each malware family .
Claims
1. A malware classification system based on the FPN-Transformer model, characterized in that, Includes the following modules: Data input module: Receives code files or code assemblies as input; Data preprocessing module: preprocesses the code data obtained from the data input module, including feature extraction, data cleaning, standardization / normalization and feature selection, to ensure the quality and consistency of the input data, and outputs the preprocessed feature data to the Hash random compression module; Hash random compression module: Uses hash algorithm to randomly map and compress preprocessed feature data into hash space; Feature selection module: Based on the set threshold, the feature data mapped and compressed by the hash algorithm is filtered, i.e., feature selection is performed to retain features that make a significant contribution to the classification of malicious code families and generate a compressed feature set; FPN-Transformer Model Module: Utilizes the FPN-Transformer model to perform in-depth analysis and feature extraction on the feature data output by the feature selection module; Multi-level regression feature extraction model module: The multi-level regression feature extraction model is used to extract multi-scale feature information from the feature data output by the FPN-Transformer model, generate channel attention features and spatial attention features and fuse them, use a linear layer to perform linear transformation on the fused features, and input the linearly transformed features into the regression layer to generate the autocorrelation matrix. Classification result output module: Based on the autocorrelation matrix obtained from the multi-level regression feature extraction model, determine whether there is malicious code in the code file, obtain the malicious code family to which the malicious code belongs, and output the final classification result.
2. The malware classification system based on the FPN-Transformer model according to claim 1, characterized in that, The FPN-Transformer model specifically includes multiple average pooling layers, M encoders, and M decoders, where M is the number of encoders. The encoder includes an input embedding layer, a multi-head attention unit, a normalization layer, a feedforward neural network, and a normalization layer connected in sequence. The structure of the decoder is the same as that of the encoder. Multi-head attention units with self-attention mechanisms and feedforward neural networks are used for feature transformation and enhancement.
3. The malware classification system based on the FPN-Transformer model according to claim 2, characterized in that, The multi-layer regression feature extraction model specifically includes a parallel channel attention module and a spatial attention module, a linear layer and a regression layer, and an autocorrelation matrix classifier. The channel attention module includes an input embedding layer, a parallel average pooling layer and a max pooling layer, and a shared multi-layer perceptron. The spatial attention module includes an input embedding layer, parallel average pooling layers and max pooling layers, and convolutional layers; The channel attention module and spatial attention module are combined with linear and regression layers to capture multi-scale information of malicious code features. The shared multilayer perceptron of the channel attention module is combined with the regression layer to enhance the richness and robustness of feature representation. Autocorrelation matrix classifiers use autocorrelation matrices to evaluate and strengthen the relationships between classification features, optimizing feature combinations through autocorrelation to improve classification accuracy and robustness.
4. The method for a malware classification system based on the FPN-Transformer model as described in claims 1-3, characterized in that, Includes the following steps: Step 1: Data preprocessing. Use a natural language processing model to extract preliminary feature data from the code files to be classified, capturing the semantic and structural information of the code; Step 2: Perform hash random compression on the feature data obtained in Step 1 to reduce the dimensionality of the feature data and reduce noise; Step 2.1: Use a hash function to map the preprocessed feature values to the hash function space; Step 2.2: Select important features based on preset thresholds to reduce the influence of irrelevant features; Step 3: Use the FPN-Transformer model to perform in-depth analysis and feature extraction on the compressed feature data; Step 4: The feature data obtained in Step 3 is further fused and classified using a multi-level regression feature extraction model; Step 5: Based on the output of the multi-level regression feature extraction model, determine whether there is malicious code in the code file, and determine the malicious code family to which the malicious code belongs.
5. The method according to claim 4, characterized in that, Step 1 specifically includes the following sub-steps: Step 1.1: Use the CodeBERT model to extract feature data from the code file, including Abstract Syntax Tree (AST), Node, Code, and Entropy; Step 1.2: Remove noise and irrelevant information from the feature data, and delete null or outlier values; Step 1.3: Standardize or normalize the feature data to ensure comparability between different features; the formula used for normalization is as follows, where... The mean, Standard deviation: ； In the formula The mean, Standard deviation, , These are the feature data before and after normalization, respectively. Step 1.4: Select feature data that is useful for the classification task and remove redundant feature data.
6. The method according to claim 5, characterized in that, Step 2.1 specifically includes: Step 2.1.1: Select a suitable hash function for data preprocessing. The selected hash function should have efficient mapping and low collision characteristics. Step 2.1.2: Transfer the feature data obtained in Step 1 Divided into multiple small pieces The hash function selected in step 2.1.1 is used to process each small block. Random mapping and compression are performed, and the hash function is: ； In the formula h Represents a hash function. i is a feature vector. i is a random vector. It is a prime number. The size of the hash space; d Indicates the size of the feature dimension.
7. The method according to claim 6, characterized in that, Step 2.2 specifically includes: Step 2.2.1: Set the threshold σ based on the feature value distribution and feature importance metric; Step 2.2.2: Retain features below the threshold σ and discard other features. The calculation formula is: ； In the formula This represents the hash feature data obtained through filtering.
8. The method according to claim 7, characterized in that, Step 3 specifically includes: Step 3.1: Convert the input sample feature data into embedding vectors; Step 3.2: Perform average pooling operations of different magnitudes on the embedded vectors to generate multiple sets of feature mapping data; Step 3.3: Concatenate the multiple sets of feature mapping data obtained in Step 3.2 along the feature dimension to form a feature patch; Step 3.4: Input the feature patch obtained in Step 3.3 into the encoder for processing; Step 3.4.1: Calculate the correlation between input features using a multi-head attention mechanism unit to generate an attention weight matrix; Step 3.4.2: The feedforward neural network performs a nonlinear transformation on the features output by the multi-head attention mechanism unit; Step 3.4.3: Normalize the feature data output by the feedforward neural network; Step 3.5: Input the feature data output by the encoder into the decoder for processing. The data processing process of the decoder is the same as in step 3.4, to obtain multi-level and multi-view feature data.
9. The method according to claim 8, characterized in that, Step 4 specifically includes: 1) Use the channel attention module to extract multi-scale feature information from the feature data output by the FPN-Transformer model to generate channel attention features; 1.1) The input feature representation is converted into an embedding vector through the input embedding layer; 1.2) Use average pooling (AvgPool) and max pooling (MaxPool) operations to generate two sets of feature maps; 1.3) Input the pooled features into a shared multilayer perceptron for feature transformation; 1.4) The features transformed by the shared multilayer perceptron are fused through element-wise addition and element-wise multiplication operations to generate channel attention features; 2) Use the spatial attention module to extract multi-scale feature information from the feature data output by the FPN-Transformer model to generate spatial attention features; 2.1) Use an input embedding layer to convert the input feature representation into an embedding vector; 2.2) Two sets of feature maps are generated using average pooling (AvgPool) and max pooling (MaxPool) operations; 2.3) Input the pooled features into the convolutional layer Conv for feature convolution operation; 2.4) The convolutional features are fused using element-wise addition and element-wise multiplication operations to generate spatial attention features; 3) Fuse the channel attention features and spatial attention features to generate the final attention feature representation; use a linear layer to perform a linear transformation on the fused features; input the linearly transformed features into a regression layer to generate an autocorrelation matrix.
10. The method according to claim 9, characterized in that, The mathematical formula for the autocorrelation matrix classifier is: ； In the formula This represents the initialization parameters in the tag space. This represents a parameter in another dimension of the label space. Represents the Kronecker delta function. Labels used to distinguish different categories and It represents the similarity or angular relationship between two vectors in the label space in a higher-dimensional space.