Malware texture feature classification method and system fusing knowledge distillation and sparse convolution

Abstract

The invention belongs to the field of computer security, and discloses a knowledge distillation and sparse convolution fused malicious software texture feature classification method and system, and the method comprises the steps: carrying out the knowledge migration from a large-scale teacher network to a light-weight student network through employing a knowledge distillation strategy; adding an attention module and a cavity space pyramid pool in the network, and extracting depth features of the malicious software image; and integrating the depth features by using a random forest as a final classifier to realize malicious software family classification. According to the method, an effective data set is generated by adopting multi-dimensional feature fusion, a feature extraction process is improved, a knowledge distillation technology is combined to optimize a model training process, and the accuracy of a classification task is improved by adopting an integrated learning classifier. According to the method, malicious software can be classified more accurately and quickly, malicious attacks can be responded and defended in time, and user data and privacy are protected; the model is high in adaptability and can adapt to malicious software variants, and the defense capability on unknown malicious software is enhanced.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of network security technology, and in particular relates to a malware texture feature classification method and system integrating knowledge distillation and sparse convolution. Background Art

[0002] As the main source of information sharing between different nodes, the Internet is often regarded as the main target of attack by cyber criminals; and the exponential growth of malware such as worms, viruses and Trojan horses has become one of the major threats to Internet security. According to the latest statistical data report from Kaspersky Lab, the number of new malware detected by the agency from around the world every day in 2022 exceeded 400,000. According to Cybersecurity Ventures' forecast, by the end of 2024, the loss of the global economy caused by cyber attacks is expected to reach 10.5 trillion US dollars, so the effective detection of malware is of great practical significance. Malware editors often use code obfuscation, subroutine rearrangement and other means to generate a large number of malware variants, which also makes how to efficiently and accurately classify malware a challenging task.

[0003] With the development of artificial intelligence, traditional signature-based or heuristic methods can no longer meet the current malware detection needs. Deep learning has attracted much attention due to its ability to learn from data and adapt to new patterns. At the same time, studies have shown that different malware from the same family have similar texture distributions when converted into images. Therefore, more and more scholars are considering further converting the malware classification problem into an image classification problem combined with deep learning for in-depth research. Cui et al. converted malicious code into grayscale images and automatically extracted and classified features of images based on convolutional neural networks (CNN). Vasan et al. processed the grayscale images generated by malware binary files into color images through color mapping technology and fine-tuned the CNN architecture to classify malware. Xing X et al. used autoencoders to classify malware represented in the form of grayscale images and proved the effectiveness of algorithm detection on Android datasets. Dhanya KA et al. generated Markov images based on the bytecode of Dalvik executable files and combined the CNN framework to detect IoT Android malware. The accuracy of distinguishing malware from benign software reached 99.41%. Xuan Bona et al. used a combination of multiple features to generate RGB (Red, Green, Blue) images, improved the CNN structure, and achieved malware classification. Although visualizing images can omit feature engineering steps and simplify the algorithm detection process, most existing studies are based on simple binary byte streams and use a single feature to convert malware into grayscale images, ignoring the contextual semantic information between malicious codes and the correlation and dependency between different features, making it difficult to effectively detect malware variants. In addition, when using deep neural network algorithms for image classification, we often face the problem of non-uniform input image size, which requires normalization, resulting in the loss or redundancy of some feature information, making the detection algorithm perform poorly. Most detection models are dedicated to improving accuracy, and are usually large in scale, making them difficult to deploy in resource-limited environments.

[0004] In view of the above analysis, the technical problems that need to be solved urgently in the prior art are:

[0005] ① Existing algorithms are difficult to reflect the contextual semantic information between malicious codes and the correlation between similar variants, and have low accuracy in identifying malware variants; ② The dataset used is large in scale, with high feature dimensions and complexity; ③ The model has large resource requirements and high time and space complexity, which makes the model generally applicable in practice. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a malware texture feature classification method and system integrating knowledge distillation and sparse convolution.

[0007] The present invention is implemented by a malware texture feature classification method that integrates knowledge distillation and sparse convolution, and the method specifically includes:

[0008] S1: Combines raw binary byte streams, assembly opcodes, and APIs to generate a GMA image dataset with rich texture features;

[0009] S2: Apply knowledge distillation strategy to transfer knowledge from the large teacher network to the lightweight student network;

[0010] S3: Add attention module and dilated spatial pyramid pool to the network to extract deep features of malware images;

[0011] S4: Using random forest as the final classifier to integrate deep features and achieve malware family classification.

[0012] Furthermore, in S1, the three types of features are visualized separately according to their respective characteristics, and finally merged into a three-channel RGB image.

[0013] Furthermore, S2, with the help of knowledge distillation strategy, selects the ResNeXt pre-trained model as the teacher network and the lightweight neural network MobileNetV2 pre-trained model as the student network.

[0014] Furthermore, the attention module in S3 introduces an efficient multi-scale attention module (EMA) into the CNN architecture, which effectively captures the dependencies between dimensions, further extracts feature information of each scale from both spatial and positional aspects, increases the receptive field, and is more efficient in determining the required parameters, thereby reducing computational overhead.

[0015] Furthermore, the atrous spatial pyramid pooling (ASPP) in S3 is composed of a combination of atrous convolution and spatial pyramid pooling layers.

[0016] Furthermore, in S4, the softmax layer in the fully connected layer of the deep learning model is modified into a flattening layer, and the extracted deep features are input into the random forest for final classification.

[0017] Another object of the present invention is to provide a malware texture feature classification system integrating knowledge distillation and sparse convolution, including two parts: deep feature extraction and malware classification, specifically including:

[0018] Knowledge distillation module, used to transfer knowledge from a large teacher network to a lightweight student network;

[0019] Atrous spatial pyramid pooling: Through the atrous spatial pyramid pooling layer, the classification network removes the restriction on the size of sample images and reduces information loss.

[0020] An efficient multi-scale attention module with parallel substructures that enables the network to handle both long-term and short-term dependencies;

[0021] Random Forest,Use Random Forest as the final classifier to integrate deep features and achieve malware family classification.

[0022] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0023] First, generate a more effective image dataset GMA: generate three sets of grayscale images, GLCM images, Markov images, and API images, respectively, map them to three channels, and merge them to obtain the RGB image dataset GMA. Statistical analysis methods are introduced in the image generation process. The fusion of multiple features not only retains the contextual semantic information in feature expression, but also reflects the dependency between different features and the association information between variants of the same type. In addition, the merging of RGB datasets also reduces the size of the grayscale image dataset.

[0024] Research on model optimization and simplification: Based on the efficient multi-scale attention mechanism and the hollow space pyramid pool, the deep neural network architecture is optimized to further obtain the deep information of features and avoid the information loss or redundancy caused by image size normalization. The knowledge distillation strategy is applied to transfer knowledge from the large-scale teacher network to the lightweight student network, which realizes the simplification of the model, effectively combining the efficiency and lightweight of deep learning, and improving the practical applicability of the model.

[0025] Improve classification accuracy and efficiency: By using the parallel computing characteristics of random forests, we can improve the problem of large computing resource requirements and high time complexity of deep learning models, increase the processing speed of the model, and thus improve the efficiency of the malware classification model. At the same time, through the built-in model integration mechanism of random forests, we can further enhance the stability and generalization ability of the model and improve the classification accuracy of the model.

[0026] Second, as auxiliary evidence of the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0027] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0028] The malware classification system is applied to downstream tasks to achieve rapid identification of malware variants. The present invention has been applied to malware defense systems in cooperation with enterprises. As a component of enterprise network protection, it improves the detection capabilities of network intrusion detection systems (IDS) and antivirus software, and provides threat intelligence analysis services based on malware classification results, helping enterprises understand network threat trends and improve their overall security defense capabilities. It has great commercial value and application value.

[0029] (2) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0030] The present invention converts grayscale symbiotic images, opcode Markov images, and API images into RGB data sets, integrating multiple features of malware, greatly improving the ability to describe malware behavior and data patterns, which is difficult to achieve with traditional single feature or text-based analysis methods. This system achieves near-real-time threat detection and response capabilities by compressing models and optimizing computing processes, filling the gap in low-latency and efficient threat response in the industry.

[0031] (3) The technical solution of the present invention solves the technical problems that people have been eager to solve but have never been able to solve successfully:

[0032] In the field of information system security, traditional high-precision models usually require a lot of computing power, which makes it difficult to meet the needs of reducing computing resources while ensuring high accuracy. Through knowledge distillation technology, the system successfully compresses complex models into lightweight models, greatly reducing the consumption of computing and storage resources while maintaining high detection accuracy.

[0033] (4) The technical solution of the present invention overcomes technical prejudice:.

[0034] Deep learning effectively captures complex nonlinear relationships in data through multi-level structures, and can generate rich feature representations, especially when processing complex data such as images. However, end-to-end deep learning models often require a lot of computing resources, and due to the large number of parameters, they are prone to overfitting, resulting in reduced generalization performance. In contrast, random forests can effectively reduce the overfitting problem through their built-in model integration mechanism, and have good robustness and generalization capabilities. In addition, random forests are insensitive to the scale and distribution of features, are easy to implement and tune, have low computing resource requirements, and are fast in training and prediction. They can effectively reduce the computational burden of deep learning models in the classification stage, making them an ideal classifier choice after feature extraction.

[0035] The present invention combines the two, fully utilizing the advantages of deep learning models in feature extraction, and reducing the demand for computing resources and the risk of overfitting by using random forests, thereby achieving a balance between performance and resource consumption, and constructing a classification model with excellent performance and good generalization ability.

[0036] Third, the technical solution of the present invention solves multiple problems in the prior art in industrial applications and has achieved significant technical progress, which is specifically reflected in the following aspects:

[0037] 1. Problems with existing technologies

[0038] (1) Insufficient feature extraction

[0039] Most existing malware detection methods are based on static features and traditional feature engineering, which are difficult to capture the complex behavior patterns and subtle texture features of malware. Static feature extraction is limited by manual design, resulting in low classification accuracy of malware families, especially when facing obfuscated or variant malware, the existing static features are often unable to distinguish their subtle differences.

[0040] (2) The model is too complex and the deployment cost is high

[0041] Although deep learning models perform well in tasks such as image recognition, their complex network structure and large number of parameters limit their application in resource-limited embedded or real-time malware detection systems. Large models have long inference time and high energy consumption, making it difficult to meet the real-time requirements in practical applications.

[0042] (3) The simplicity and limitations of traditional classifiers

[0043] In existing classification methods, a single classifier is usually used to classify malware. This method cannot effectively integrate information at multiple levels and has difficulty dealing with diverse malware samples, resulting in limited classification performance.

[0044] 2. Technical advancement of the present invention

[0045] (1) Rich malware texture feature extraction

[0046] By combining raw binary byte streams, assembly opcodes, and API call information, a GMA image dataset with rich texture features is generated. Compared with traditional manual feature extraction methods, this method can more intuitively and comprehensively characterize the behavior patterns of malware, especially when dealing with complex malware families. This method effectively improves the expressiveness of features and significantly improves classification accuracy.

[0047] (2) Knowledge distillation improves model efficiency

[0048] Through the knowledge distillation technology, knowledge is transferred from the large teacher network to the lightweight student network, which greatly reduces the computational complexity of the model while ensuring the classification accuracy. This enables the model proposed in the present invention to run efficiently in a resource-constrained environment, suitable for embedded devices or real-time malware detection scenarios, and solves the problem of high model complexity in the prior art.

[0049] (3) Combining the Attention Mechanism with Dilated Spatial Pyramid Pooling

[0050] The present invention introduces the attention mechanism and the Atrous Spatial Pyramid Pooling (ASPP) module into the neural network, which enables the network to focus on the key areas of the malware image and extract global and local information through multi-scale receptive fields. This combination greatly enhances the feature extraction effect in malware classification, enables the classification model to more accurately capture the subtle differences of malware, and improves the robustness and generalization ability of classification.

[0051] (4) Random forest ensemble classifier improves accuracy

[0052] After feature extraction, random forest is used as the final integrated classifier to improve the accuracy and stability of malware family classification by fusing multiple classification results. Random forest has good integrated learning capabilities and can provide better performance than a single classifier when processing complex multi-dimensional features, significantly improving the accuracy and reliability of detection.

[0053] 3. Significant progress in industrial applications

[0054] The technical solution of the present invention combines deep learning with traditional machine learning, which not only solves the problems of insufficient feature extraction and high model complexity in the prior art, but also significantly improves the accuracy, efficiency and deployment cost of the model in practical applications by introducing knowledge distillation, attention mechanism and random forest classifier. This innovative solution has important industrial application value in the fields of malware detection, network security, embedded device security protection, etc., and helps to improve the intelligence level of existing security detection systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of a malware texture feature classification method integrating knowledge distillation and sparse convolution provided by an embodiment of the present invention;

[0056] Figure 2 This is a gray level co-occurrence matrix image generation process provided by an embodiment of the present invention;

[0057] Figure 3 It is a feature visualization of the assembly opcode provided by an embodiment of the present invention;

[0058] Figure 4 It is a feature visualization of an API call sequence provided by an embodiment of the present invention;

[0059] Figure 5 It is the three-channel feature image merging provided by the embodiment of the present invention;

[0060] Figure 6 It is the EMA core framework provided by the embodiment of the present invention;

[0061] Figure 7 This is a diagram of the architecture of a malware texture feature system integrating knowledge distillation and sparse convolution provided by an embodiment of the present invention;

[0062] Figure 8 It is the effectiveness evaluation of the multimodal visualization data set provided by the embodiment of the present invention;

[0063] Fig. 9 is a model confusion matrix provided by an embodiment of the present invention;

[0064] Fig.10 It is an experimental evaluation of the classification algorithm provided by the embodiment of the present invention;

[0065] Fig.11 It is a diagram of the architecture of the malware detection system of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0067] like Figure 1 As shown, an embodiment of the present invention provides a malware texture feature classification method integrating knowledge distillation and sparse convolution, and the method specifically includes:

[0068] S1: Combines raw binary byte streams, assembly opcodes, and APIs to generate a GMA image dataset with rich texture features;

[0069] S2: Apply knowledge distillation strategy to transfer knowledge from the large teacher network to the lightweight student network;

[0070] S3: Add attention module and dilated spatial pyramid pool to the network to extract deep features of malware images;

[0071] S4: Using random forest as the final classifier to integrate deep features and achieve malware family classification.

[0072] The malware texture feature classification method provided by the present invention combines knowledge distillation and sparse convolution to generate high-dimensional texture features by combining raw binary data, assembly opcodes and APIs, and achieves efficient classification by combining deep learning with traditional machine learning. The specific working principle is as follows:

[0073] First, in step S1, the malware's original binary byte stream, assembly opcodes, and API call information are converted into GMA (General Malware Analysis) images through a special mapping method. These images can show the behavior patterns and operation logic of malware in a two-dimensional form, thus forming an image dataset with rich texture features. Through this data enhancement method, the malware's structural information is intuitively represented in the image, providing a solid foundation for subsequent feature extraction and classification.

[0074] Next, in step S2, a knowledge distillation strategy is applied. By training a large-scale teacher network, a deep understanding of malware features is obtained and this knowledge is transferred to a lightweight student network. This knowledge transfer enables the student network to efficiently learn complex features through soft labels and intermediate layer feature matching, while reducing the complexity of the model, making it suitable for real-time malware classification in resource-constrained environments.

[0075] In step S3, the attention module and Atrous Spatial Pyramid Pooling (ASPP) are combined. Through the sparse convolutional network structure and the attention mechanism, the network can focus on the key areas of the malware image and enhance the ability to extract important features. At the same time, the Atrous Spatial Pyramid Pooling module extracts deep features through receptive fields of different scales, effectively capturing the global and local information in the malware image and improving the richness of feature expression.

[0076] Finally, in step S4, the extracted deep features are integrated and classified through the random forest classifier. The integrated learning ability of random forest enables it to process complex multi-dimensional features and combine the features extracted by the deep network to achieve efficient classification of malware families. Through this multi-module collaboration, the system can quickly and accurately identify malware and improve classification performance and accuracy.

[0077] The malware texture feature classification method provided by the embodiment of the present invention integrates knowledge distillation and sparse convolution, mainly generates texture feature images from the binary stream and API call of the malware, and uses a deep learning model for classification. Its working principle can be divided into the following four parts:

[0078] Step S1: Generate a GMA image dataset with rich texture features

[0079] First, the original binary byte stream, assembly opcodes and API call information of the malware are combined and converted into image data. This step extracts the low-level behavior information of the malware and converts it into a pixel matrix to generate a GMA (Grey level co-occurrence, Markov, API) image dataset with texture features. The purpose of this is to express the relationship between the binary information of the original malicious file, the global information, the local information of the assembly instructions, and the malicious code context in the form of an image, making the structural pattern of the malware more obvious and laying the foundation for subsequent image processing and classification.

[0080] Step S2: Knowledge transfer in knowledge distillation strategy

[0081] In order to improve the training efficiency of the model, a knowledge distillation strategy is used. Through knowledge distillation, a large and complex teacher network is first trained, which has better performance in classification accuracy and feature extraction. Next, the features and classification capabilities learned in the teacher network are "distilled" into a lightweight student network, so that the student network maintains low computational complexity while retaining high classification performance. This process can reduce training time and model size while ensuring that the lightweight student network can effectively extract malware features.

[0082] Step S3: Deep feature extraction and enhancement

[0083] To further improve the accuracy of malware image classification, attention modules and Atrous Spatial Pyramid Pooling (ASPP) are added to the network. The attention module is used to capture more important areas in the image, ensuring that the network can focus on the most representative texture features in the malware image. Atrous Spatial Pyramid Pooling expands the receptive field through multi-scale atrous convolutions, thereby capturing global and local information of the malware image. Combining these two modules can significantly enhance the network's feature extraction capabilities, making the network more accurate in capturing malware family features.

[0084] Step S4: Final classification by random forest classifier

[0085] After extracting rich deep features, random forest is used as the final classifier. Random forest is an ensemble learning method that classifies malware families by building multiple decision trees and taking majority votes of their classification results. Since random forest has strong robustness in processing high-dimensional data and complex features, it can effectively integrate deep features and improve classification accuracy. At the same time, it can avoid overfitting problems, so that the entire system has better generalization capabilities when facing different types of malware.

[0086] Among the commonly used features, the binary sequence can retain the binary information and global information of the original malicious file, but ignores the semantic information between specific malicious codes; the opcode sequence can retain some information of the assembly instructions in the malware, and the API call function can reflect the relationship between the malicious code context. Therefore, the three types of features are visualized according to their respective characteristics, and finally merged into a three-channel RGB image for subsequent in-depth research and classification. Including:

[0087] (1) Visualization of raw binary byte stream

[0088] Gray Level Co-occurrence Matrix (GLCM) is a statistical analysis method that uses grayscale to describe spatial texture features. It can reflect the comprehensive information of grayscale in terms of direction, adjacent intervals, and change amplitude, but the one-dimensional vector it generates does not meet the input requirements of CNN. Therefore, the present invention directly uses the grayscale co-occurrence matrix and the texture and spatial information stored therein as a grayscale matrix image compatible with it. In order to avoid the loss of basic feature information, the single-channel grayscale co-occurrence matrix image (B) is converted from the original byte binary stream of the PE file. The process is as follows: Figure 2 shown.

[0089] a. Grayscale image generation

[0090] First, the original binary byte stream sequence is divided into 8-bit substrings (b7, b6, b5, b4, b3, b2, b1, b0), and each substring is converted into an unsigned integer in [0, 255] and saved as a pixel point p in a single-channel image. bin The conversion formula is:

[0091] p bin =b7×2 7 +b6×2 6 +b5×2 5 +b4×2 4 +b3×2 3 +b2×2 2 +b1×2 1 +b0×2 0 (1)

[0092] This operation can construct a one-dimensional feature vector containing the decimal value of the 8-bit substring. Arrange the pixel values ​​in order and fill the two-dimensional matrix to generate the required grayscale image. Files from different malware families vary, and the size of the generated image is also different. Its height depends on the content of the PE file, while the width is determined by the size of the PE file.

[0093] b. Histogram equalization and grayscale image discretization

[0094] The converted grayscale image is subjected to histogram equalization to enhance the local contrast of the image. Then, the grayscale image is discretized from 256 to 128 levels to quickly speed up the GLCM generation process.

[0095] c. GLCM image generation

[0096] The GLCM is calculated based on the discretized 128-level grayscale image. Formally, for each distance and angle value, each matrix contains a sub-matrix of size 128×128, which contains texture and spatial information. Since the distance value used in the present invention is 1, the calculated GLCM can be reshaped into 4 separate 128×128 sub-matrices of different angle values. By stacking these 4 matrices together, a final image of size 256×256 is formed, namely the GLCM image.

[0097] (2) Assembly Instruction Visualization

[0098] Assembly opcodes can reflect the behavior of a program, so malware from the same family usually has similar opcode sequences. In order to effectively summarize opcodes, consider encoding opcodes of the same type in the same way. For example, MOV, MOVQ, and MOVD used for general data transmission functions are merged into the same opcode. After screening and merging opcodes with the same functions, the opcode range is finally fixed at 255. The remaining opcodes are classified as the 256th due to their extremely small proportion (about 0.04%), so the encoding range is 0 to 255. In order to avoid missing useful information due to discarding bytes or opcodes and to reduce the sparsity of the generated image, the present invention uses the transition probabilities of various opcodes extracted from the assembly instruction sequence to generate a Markov single-channel image (G). The transition probability of each opcode is calculated in sequence to obtain a one-step transition probability matrix, and the Markov single-channel image (G) of the assembly opcode is generated based on this. The visualization process is as follows Figure 3 .

[0099] The following describes the calculation method of the transition probability and the one-step transition probability matrix. Assuming that the next state is only related to the current state, the assembly instruction sequence can be regarded as a Markov chain, that is, P(t i+1 |t0,…,t i )=P(t i+1 |t i ). Assuming that the previous state m transfers to the next state n with a certain probability, the transition probability P is calculated using formula (2) m,n .

[0100]

[0101] There exists n transition probability matrices with n states. 2 The transition probability matrix is ​​an n×n matrix. Therefore, the generated transition probability matrix is ​​shown in formula (3), which can be directly used as a two-dimensional tensor to generate single-channel image pixels, with a size of 256×256 and uniform size.

[0102]

[0103] Among them, the element p in the i-th row and j-th column of M i,j represents the transition probability from state i to state j.

[0104] (3) API call sequence visualization

[0105] In malware, a series of related Windows Application Programming Interface (API) call sequences are important representations of its behavioral attributes, and are therefore considered to be one of the most representative features in the field of malware detection. The API call sequence can be regarded as a contextual representation of a calling program. In order to extract this contextual semantic relevance, the present invention classifies similar APIs into the same category based on a similarity clustering method, and corresponds to a limited number of dynamic libraries. According to the provided .asm disassembly file, the API call sequence of the entire data set sample is extracted according to the internal order of the dynamic libraries in various pieces of the present invention and the external call relationship.

[0106] If punctuation marks are included as part of the input, it may cause difficulties in the model learning stage. Therefore, such irrelevant information is ignored during extraction, and only the ASCII value of the grouped single API cluster name is converted alphabetically. After conversion, the obtained encoding range is within [0,255], and the pixel value of the corresponding single-channel image (R) can be directly generated from it. An example of API encoding is shown in Table 1. Figure 4 Feature visualization flow for API call sequences.

[0107] Table 1 API coding example

[0108]

[0109] (4) Three-channel image merging

[0110] Based on the OpenCV library, the three sets of single-channel grayscale images formed by the visualization of the above three types of features are mapped into three channels, thereby obtaining a multi-view RGB image dataset with rich texture information, denoted as GMA. Using GMA as the input of the MTFCD algorithm will enable the network to learn richer texture information including contextual semantic relevance. Figure 5 Examples of feature visualization and RGB image generation are given.

[0111] Since the existing public datasets do not contain all the latest malware resources, many new data are not open to the public. At the same time, overfitting may occur when training malware detectors. Transfer learning uses the knowledge learned from the source domain to solve the problems of the target domain, which can alleviate the problems of insufficient computing resources and unbalanced datasets in detection. Therefore, the selected networks are all pre-trained, and the parameters are fine-tuned for malware detection tasks.

[0112] In order to reduce the model size and reduce the computational complexity and parameter complexity of the detection algorithm, S2, with the help of knowledge distillation strategy, selects the ResNeXt pre-trained model as the teacher network and the lightweight neural network MobileNetV2 pre-trained model as the student network. Compared with the ResNet model commonly used for malware image texture detection, the ResNeXt network introduces group convolution technology on its basis, which is conducive to enhancing model accuracy and scalability, and ensuring computational efficiency. MobileNetV2 uses deep separable convolution with an inverted residual structure to optimize training speed and parameter efficiency, further improving the detection accuracy and speed of the model. The model algorithm is as follows:

[0113]

[0114] The attention module in S3 introduces the efficient multi-scale attention module (EMA) into the CNN architecture, effectively capturing the dependencies between dimensions, further extracting feature information of each scale from the aspects of space and position, increasing the receptive field, and being more efficient in determining the required parameters and reducing computational overhead. The core framework of EMA is as follows: Figure 6 shown.

[0115] The multi-scale attention module adopts a parallel substructure to save the network from more sequential processing and avoid falling into greater depth. For the input malware feature map, EMA divides it into multiple sub-features in the channel dimension to learn different semantics. In the 1×1 branch, the sub-features are subjected to one-dimensional global average pooling in the width and height directions respectively, and then the results in the two directions are concatenated and mapped to the Sigmoid nonlinear activation layer respectively through the 1×1 convolution kernel, and then the weights are recalculated and enter the cross-learning module. Only a single 3×3 convolution kernel is stacked in the 3×3 branch to capture the local multi-scale feature representation. The cross-learning module uses two tensors output by the 1×1 and 3×3 branches, and uses two-dimensional global average pooling to encode global spatial information. The output of the smallest branch will be converted to the corresponding dimensional shape directly before the joint activation mechanism of the channel features; the output tensor is fitted with a linear transformation based on the nonlinear function softmax, and the output of the parallel processing is subjected to a matrix dot product operation to obtain the first spatial attention map. Similarly, 2D global average pooling is used to encode the global spatial information in the 3×3 branch, while the output of the 1×1 branch is directly converted to the corresponding dimensional shape before the joint activation mechanism of the channel features, thereby deriving the second spatial attention map that retains the entire precise spatial position information. Finally, the two generated spatial attention weights are aggregated to obtain the output feature map, and the Sigmoid function is used to capture the pairwise relationship at the pixel level and highlight the global context information of all pixels.

[0116] After this operation, the network has the ability to process both long-term and short-term dependencies, and thus obtains global spatial weights, enabling subsequent deep learning networks to utilize a wider range of spatial features and contextual information at different scales.

[0117] The atrous spatial pyramid pooling (ASPP) in S3 is composed of a combination of atrous convolution and spatial pyramid pooling (SPP). Atrous convolution can expand the receptive field and capture multi-scale contextual information; SPP can accept image inputs of any size and output them as vectors of the required dimensions. There are multiple parallel pooling layers with different convolution rates in the ASPP structure, which ensures that feature information of different distributions can be sampled and aggregated. Therefore, the addition of ASPP can remove the restrictions on the size of sample images from the classification network and reduce the occurrence of information loss, thereby improving the classification accuracy and robustness of the model.

[0118] The imbalance in the number of malware families causes the classification results output by the classifier to tend to the category with more samples. Random Forest (RF), as an ensemble learning algorithm, has significant advantages in alleviating the problem of data imbalance, so RF is selected as the final classifier of the algorithm of the present invention. RF first needs to estimate the probability of the output of each tree and obtain the probability value P of the trth tree for each category k. tr (k|V), where V is the deep feature extracted by the deep neural network; then, calculate its average value P(k|V), as shown in formula (4), where TR is the total number of trees; finally, use the argmax function to select the most likely category from the average probability P(k|V) as the final prediction result As shown in formula (5):

[0119]

[0120] Among them, argmax k To find the category k that maximizes P(k|V).

[0121] like Figure 7 As shown, an embodiment of the present invention provides a malware texture feature classification system integrating knowledge distillation and sparse convolution, including two parts: deep feature extraction and malware classification, specifically including:

[0122] Knowledge distillation module, used to transfer knowledge from a large teacher network to a lightweight student network;

[0123] Atrous spatial pyramid pooling: Through the atrous spatial pyramid pooling layer, the classification network removes the restriction on the size of sample images and reduces information loss.

[0124] An efficient multi-scale attention module with parallel substructures that enables the network to handle both long-term and short-term dependencies;

[0125] Random Forest,Use Random Forest as the final classifier to integrate deep features and achieve malware family classification.

[0126] The following are two specific application examples, showing the actual application scenarios of the malware texture feature classification method integrating knowledge distillation and sparse convolution:

[0127] Example 1: Malware Detection and Classification at a Financial Institution

[0128] A large financial institution is often attacked by malware targeting its payment system and customer data. In order to improve the defense capability of network security, the institution decides to adopt the malware texture feature classification method of the present invention to enhance its malware detection and classification system.

[0129] Application steps:

[0130] 1. Data collection and preprocessing: Financial institutions extract a large number of binary files, API calls, and assembly instructions from their network traffic and known malware libraries. According to step S1 of the present invention, these data are converted into a GMA image dataset with rich texture features.

[0131] 2. Model training and knowledge distillation: To ensure the efficiency and accuracy of the detection system, a large teacher network is first used on the server side for preliminary training. The teacher network learns the characteristics of a large number of financial malware, including Trojans, ransomware, etc. Then, through the knowledge distillation strategy, the knowledge of the teacher network is transferred to the lightweight student network embedded in the firewall. In this way, the firewall can run efficiently with a lower computing resource burden.

[0132] 3. Deep feature extraction and classification: In actual use, the system extracts deep features of malware GMA images through an embedded deep learning network. With the help of attention modules and dilated spatial pyramid pooling, the system is able to identify subtle features of malware, especially those for complex and variant malware.

[0133] 4. Real-time malware classification and response: The system uses a random forest classifier to classify the extracted features, accurately identify malware families, and distinguish between known and unknown threats. When malware is detected, the system can isolate the source of infection in real time and issue an alarm, helping financial institutions to block attacks in a timely manner and prevent financial losses and data leaks.

[0134] By using this malware detection method, financial institutions have significantly reduced losses caused by malware attacks, especially when facing attacks from a variety of new malware. The system's high detection accuracy and response speed effectively protect payment systems and user data.

[0135] Example 2: Malware Identification in Mobile Device Security Applications

[0136] A mobile security company needs to develop a lightweight mobile device security application to detect and classify malware (such as spyware, adware, Trojans, etc.) on mobile devices. Due to the limited computing resources of mobile devices, how to reduce the computing burden while ensuring detection accuracy becomes a challenge.

[0137] Application steps:

[0138] 1. Dataset construction and GMA image generation: By extracting binary files, API calls, and operation instructions from mobile devices, mobile security companies convert them into GMA image datasets according to step S1. These images represent the characteristics of various mobile malware.

[0139] 2. Teacher network training and knowledge distillation: The company used a high-performance GPU cluster on the server to train a large teacher network specifically for analyzing the texture features of malware. Subsequently, the knowledge of the teacher network was transferred to a lightweight student network in the mobile application using the knowledge distillation method, enabling the mobile terminal to efficiently perform malware detection.

[0140] 3. Real-time malware detection on mobile devices: When the mobile device is running, the student network detects the behavior of the application in real time in the background, and extracts deep features through the attention module and the void space pyramid pool to identify potential malware. Combined with the random forest classifier, the system can accurately classify the family types of various malware.

[0141] 4. User notification and quarantine: Once malware is detected, the app will immediately notify the user and provide suggestions for quarantine or uninstallation. In addition, the system will upload detected malware samples to the cloud to further improve the recognition ability of the detection model.

[0142] The mobile device security application ensures that mobile users are protected from malware threats through lightweight models and accurate detection capabilities without significantly increasing the burden on the device. User feedback shows that the application outperforms most similar products on the market in terms of malware detection accuracy and speed.

[0143] Through these two specific embodiments, the malware texture feature classification method of the present invention demonstrates its powerful practical application value in the financial industry and mobile security fields.

[0144] In order to prove the effectiveness of the classification algorithm proposed in this invention, a public malware dataset was selected for training and verification, and the results were compared with the baseline algorithm and related literature.

[0145] 1. Datasets and evaluation metrics

[0146] 1.1 Dataset

[0147] This paper selects the public dataset MMCC provided by Microsoft in the malware classification competition held on Kaggle in 2015 to evaluate the algorithm effect. MMCC consists of a training dataset and a test dataset, but since the sample labels are not provided in the test dataset, only the training dataset is used for the experiment. The training set contains 10,868 samples from 9 malware families, including .byte and .asm file formats. Table 2 shows its sample distribution.

[0148] Table 2 Distribution of samples in the MMCC dataset

[0149]

[0150] 1.2 Evaluation indicators

[0151] Due to the imbalance of the data set, this paper uses four common evaluation indicators to evaluate the classification performance of the algorithm, namely, accuracy, recall, and w ), Precision w ) and F1 score (F1 w ), and calculate according to formulas (6) to (9).

[0152]

[0153] in, n k is the number of samples of category k, N is the total number of samples; Recall k 、Precision k With F1 k They are the recall rate, precision rate and F1 score corresponding to each type of samples respectively.

[0154] (II) Experimental comparison

[0155] 2.1 Basic Experimental Settings

[0156] The GMA image dataset generated by MMCC is divided into 8:2 to obtain the training set and the test set. The hardware environment used in the experiment of this invention is AMD EPYC 965496-Core processor, NVIDIA GeForce RTX 4090 GPU; the software environment is Ubuntu 22.4, Python 3.10, and the PyTorch-2.1.0 framework is used.

[0157] In order to fully verify the generalization effect of the proposed detection framework, the experiment adopts the 10-fold cross validation method. The grid search technology is used for parameter optimization. Table 3 shows the search range of hyperparameters and the optimal parameter configuration on the test set. The batch size (Batch Size) of the deep feature extraction model is set to 64, and the training round (Epoch) is set to 60; in order to ensure the stability of the model during the learning process, the cosine decay learning rate adjustment strategy (Warm-up Cosine Annealing) with a warm-up mechanism is adopted, and the initial learning rate is 0.001. The Adam optimizer and the cross entropy loss function are used in the optimization process. The number of decision trees n_estimators in the linear classifier RF is 60.

[0158] Table 3 Hyperparameter settings

[0159]

[0160]

[0161] Taking into account the completeness of the experiment, the present invention conducts experimental comparison based on the generated dataset and the detection model framework, including dataset effectiveness evaluation, model compression comparison, ablation experiment, and classification algorithm effectiveness comparison, to prove the effectiveness of the dataset and the accuracy of the detection algorithm.

[0162] 2.2 Dataset Validity Verification

[0163] Experiments were conducted based on different malware texture visualization combinations in common malware detection networks (such as basic CNN structures, VGG16, ResNet-50, and EfficientNetV2) and the MTFCD framework proposed in this invention to evaluate the effectiveness of the GMA dataset proposed in this invention. The combination schemes of the visualization dataset are shown in Table 4.

[0164] The basic CNN sets four convolutional layers, a maximum pooling layer, a Dropout layer, and a dense layer, and uses two flat layers as fully connected layers before the final layer. In order to overcome the overhead of data processing, the present invention uses dropout and DeCov regularizers.

[0165] Table 4 Visualization dataset generation scheme

[0166]

[0167] The experimental results are compared as Figure 8 . With the gradual optimization of the visualization combination scheme, the classification accuracy of each detection network shows an overall upward trend, and all reach the highest value in the Sc4 scheme. Among them, the detection accuracy of the algorithm of the present invention is improved from 98.07% in the Sc1 scheme to 99.17% in the Sc4 scheme. This shows that in different visualization data sets, Sc4 provides the best feature representation. In addition, under each visualization scheme, the detection accuracy of the algorithm of the present invention is significantly better than other commonly used malware detection networks as a whole, which also proves its excellent detection effect and generalization effect.

[0168] Analysis shows that compared with Sc1, Sc2 uses GLCM images generated by bytecode, which not only extracts global information but also takes into account the spatial relationship between textures; Sc3 further generates Markov images based on the extracted opcode sequence, effectively utilizing the transition probability between assembly instructions to extract deeper patterns and dependencies, thus achieving better detection results. The GMA dataset combines GLCM images, Markov images, and API grayscale images, providing a more comprehensive perspective of information, effectively enhancing the model's adaptability to malware variants and improving the robustness of the algorithm. Overall, the GMA dataset has achieved better performance in each detection network and is more effective than other visualization combination solutions.

[0169] 2.3 Model Compression Comparison

[0170] To verify the impact of knowledge distillation technology on model performance, the parameters, floating point numbers, running time, and accuracy when using ResNeXt-50 and MobileNetV2 models for direct training and MobileNetV2 as a student model are compared, as shown in Table 5. The parameter quantity is used to measure the model scale, the floating point number is an indicator for measuring the model computing resource consumption, and the running time is used to reflect the time complexity of the model.

[0171] Table 5 Model compression comparison

[0172]

[0173] The results show that compared with ResNeXt-50 as a teacher model, the lightweight MobileNetV2 model has a smaller model size and significantly shortened running time, whether it is directly trained or used as a student model. Compared with the EfficientNetV2 model, its parameter volume is 22.1×106 and the floating point number is 8.42×108; the parameter volume and floating point number of the MobileNetV2 model used in the present invention are smaller, that is, the computing resource consumption is smaller. Moreover, the accuracy of the MobileNetV2 used for the student model is improved by 3.5% compared with the directly trained model. It can be considered that the knowledge distillation strategy applied by the present invention enables a simple network structure to learn the feature expression of a complex network, thereby showing superior model performance while consuming less resources.

[0174] 2.4 Ablation Experiment

[0175] In order to study the influence of other modules on the MTFCD framework of the present invention, ablation experiments are designed to evaluate the effectiveness of the model. EMA, ASPP and the final classifier are removed respectively, and the MTFCD algorithm is set as the control group to implement 4 groups of ablation experiments. Fig. 9The experimental results and confusion matrix are given respectively.

[0176] Table 6 Ablation experiment results evaluation

[0177]

[0178] Analysis Table 6 and Fig. 9 It can be seen that compared with the final framework MTFCD, the lack of a certain module will lead to different degrees of model performance degradation, which is reflected in the data as the reduction of the numerical values ​​of various indicators. It can be considered that the addition of the EMA mechanism enables the deep feature extractor to obtain more effective feature information from the GMA data set, guides the network to take into account the global and local information of the data set, and then adjusts the feature map weight to accelerate convergence. Due to the addition of the ASPP module, it is unnecessary to increase the data processing steps such as cutting and deformation when designing the network, and the loss and redundancy of feature information are avoided to a certain extent. The softmax layer is replaced by the RF algorithm, which effectively helps the model overcome the problem of data imbalance, reduces the impact of outliers on the MTFCD classification accuracy, and increases the robustness of the model. Therefore, thanks to the comprehensive optimization mechanism of the present invention, the MTFCD algorithm comprehensively considers the various problems that may occur in the data input, feature extraction and classification process, and performs best in various evaluation indicators.

[0179] 2.5 Evaluation of the effectiveness of classification algorithms

[0180] In order to evaluate the effectiveness of the final classifier in malware detection, K-Nearest Neighbors (KNN), Logistic Regression, eXtreme Gradient Boosting (XGBoost), Support Vector Machine (SVM) and the RF algorithm used in the present invention are selected and combined with different main network structures, and experimental evaluation is performed based on default parameters to compare the performance. Fig.10 The experimental results show that RF can achieve higher accuracy when combined with each network architecture. Therefore, RF performs better in malware detection tasks.

[0181] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0182] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A malware texture feature classification method integrating knowledge distillation and sparse convolution, characterized in that: The method specifically includes: S1: Combines raw binary byte streams, assembly opcodes, and APIs to generate a GMA image dataset with rich texture features; S2: Apply knowledge distillation strategy to transfer knowledge from the large teacher network to the lightweight student network; S3: Add attention module and dilated spatial pyramid pool to the network to extract deep features of malware images; S4: Using random forest as the final classifier to integrate deep features and achieve malware family classification.

2. The malware texture feature classification method integrating knowledge distillation and sparse convolution as claimed in claim 1, characterized in that: In S1, the three types of features are visualized separately according to their respective characteristics, and finally merged into a three-channel RGB image.

3. The malware texture feature classification method integrating knowledge distillation and sparse convolution as claimed in claim 1 is characterized in that: The S2, with the help of knowledge distillation strategy, selects the ResNeXt pre-trained model as the teacher network and the lightweight neural network MobileNetV2 pre-trained model as the student network.

4. The malware texture feature classification method integrating knowledge distillation and sparse convolution as claimed in claim 1, characterized in that: The attention module in S3 introduces the efficient multi-scale attention module (EMA) into the CNN architecture, effectively captures the dependencies between dimensions, further extracts feature information of each scale from the aspects of space and position, increases the receptive field, and is more efficient in determining the required parameters, thereby reducing computational overhead.

5. The malware texture feature classification method integrating knowledge distillation and sparse convolution as claimed in claim 1, characterized in that: The atrous spatial pyramid pooling (ASPP) in S3 is composed of a combination of atrous convolution and spatial pyramid pooling layers.

6. The malware texture feature classification method integrating knowledge distillation and sparse convolution as claimed in claim 1, characterized in that: In S4, the softmax layer in the fully connected layer of the deep learning model is modified into a flattening layer, and the extracted deep features are input into the random forest for final classification.

7. A malware texture feature classification system based on the fusion of knowledge distillation and sparse convolution as described in claims 1-6, characterized in that: The system consists of two parts: deep feature extraction and malware classification, including: Knowledge distillation module, used to transfer knowledge from a large teacher network to a lightweight student network; Atrous spatial pyramid pooling: Through the atrous spatial pyramid pooling layer, the classification network removes the restriction on the size of sample images and reduces information loss. An efficient multi-scale attention module with parallel substructures that enables the network to handle both long-term and short-term dependencies; Random Forest,Use Random Forest as the final classifier to integrate deep features and achieve malware family classification.

Images