Malicious code image classification method and system

By optimizing the convolutional neural network structure, reducing the amount of parameters and computational complexity, the problems of parameter redundancy and resource consumption in malicious code image classification are solved, and more efficient and accurate malicious code recognition is achieved.

CN118968141APending Publication Date: 2024-11-15AIR FORCE UNIV PLA
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411003652.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing convolutional neural networks (CNNs) have parameter redundancy and computational complexity in malicious code image classification, resulting in high resource consumption, reduced effectiveness in overfitting and identifying new or variant malware, and difficulty in adapting to malware. The author's technological evolution.

Method used

A malicious code image classification method is proposed. By constructing the EssicaNet model, the Stem subnet and multi-stage subnet are used for convolution processing, and the number of input channels of the convolution kernel is reduced through the kernel generation function, and the convolution blocks can be separated from each other through the kernel. Structure reparameterization, optimize network structure to reduce parameter quantity and computational complexity.

Benefits of technology

It effectively reduces the parameter quantity and computational complexity of the CNN model, improves network efficiency and performance, can more accurately identify malicious code images, adapt to the dynamic characteristics and technological evolution of malware, and is suitable for resource-constrained environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968141A_ABST
    Figure CN118968141A_ABST
Patent Text Reader

Abstract

The invention discloses a malicious code image classification method and system, relates to the field of network information security, and is used for improving the malicious code image classification efficiency. According to the method, a classification network is constructed and trained, malicious code images are classified by using a trained model, and the classification network performs first convolution processing on the input malicious code images by using a Stem sub-network; performing multi-stage second convolution processing on the feature map after the first convolution processing by using a multi-stage sub-network, and performing third convolution processing on a convolution map obtained in the previous stage by using a transition sub-network between every two adjacent stages; the number of input channels of each convolution kernel of the classification network is configured to be one, and the input channels are mapped back to full-size convolution by using a kernel generation function; and the feature map output in the last stage is classified by the classifier sub-network. According to the method, the model parameter redundancy is reduced, the model efficiency and performance are improved, the feature extraction receptive field is expanded, and the classification accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of network information security, and in particular to a malicious code image classification method and system. Background Art

[0002] As technology advances by leaps and bounds, the popularity of digital devices such as mobile phones, laptops, and webcams has increased dramatically. However, the deep integration of technology into daily life has unexpectedly exacerbated cyber threats, and the problem of security vulnerabilities has become increasingly serious. Manufacturers often ignore potential security vulnerabilities in their pursuit of product functional appeal, resulting in the market being flooded with products with inherent security flaws. Fierce competition among manufacturers and the fast pace of product development have further exacerbated this problem, with many devices with unresolved security flaws flooding the market. This provides an opportunity for malware developers, who exploit these vulnerabilities to develop malware designed to harm computer systems.

[0003] Malware is intentionally designed to perform a range of malicious behaviors including file encryption, ransomware extortion, system damage, illegal network access, data theft or destruction, etc. Its form continues to evolve and become increasingly complex, including adware, Trojans, backdoors, ransomware, spyware, and worms. According to statistics from av-atlas.org, as of March 2024, the number of recorded malware and potentially unwanted applications (PUA) has exceeded 1.37 billion, and the monthly growth of malware has exceeded 4.47 million since 2015, which highlights the severity of the problem.

[0004] The proliferation of malware poses a significant threat to Internet security, creating a dangerous situation that requires urgent attention. These malicious codes are designed to interfere with, damage or illegally gain access to computer systems, networks and their associated data. Malware can enter systems through a variety of means, such as phishing attacks, impromptu downloads or exploiting software vulnerabilities. Once inside a system, they can perform a range of malicious activities, from stealing sensitive information such as login credentials and financial data to encrypting files and demanding a ransom in the case of ransomware. This violation of privacy and confidentiality can have devastating effects on both individuals and businesses. Malware can also be used to launch distributed denial of service (DDoS) attacks, flooding target servers with fake traffic and making legitimate services inaccessible. Such attacks can damage critical infrastructure, disrupt essential services and even threaten network security. Malware can also create backdoors that allow attackers to remotely control infected systems, which not only allows attackers to execute arbitrary commands, but also helps attackers install more malware, further compromising the integrity and security of the system. This access facilitates the creation of botnets, which are networks of compromised devices that can be used for spam, data theft and even large-scale cyberattacks.

[0005] The growing threat posed by malware has highlighted the importance of cybersecurity. Organizations around the world have responded by stepping up their cybersecurity activities, recognizing that effective defense against malware requires a comprehensive understanding of its characteristics and behaviors. As a result, malware analysis has become a critical component of cybersecurity efforts. Accurate and timely detection and classification of malware is essential to combating this growing threat. Malware analysis is generally divided into two approaches: static analysis and dynamic analysis. Static analysis involves examining binary code without execution, providing a relatively fast process, but is susceptible to code obfuscation techniques. In contrast, dynamic analysis involves executing malicious code in a controlled environment and observing its behavioral patterns. While this approach is resistant to obfuscated and polymorphic malware, it is more time-consuming in comparison. Traditionally, malware detection and classification have relied on either signature-based or heuristic-based approaches, which fall under the umbrella of static analysis. Signature-based approaches use specific signatures specific to different malware families and variants to classify newly discovered malware files based on their compliance with these predefined signatures. On the other hand, heuristic-based approaches rely on rules and byte patterns developed by industry experts and analysts to extract insights from existing malware data to classify new instances. However, these approaches have limitations in coping with the dynamic landscape of modern malware.

[0006] In recent years, malware visualization has emerged as an innovative static analysis technique that provides a new perspective on the complex challenges of malware classification. This approach involves converting binary codes into grayscale images, which are then applied to machine learning algorithms for malware family classification. Malware visualization overcomes some of the limitations of traditional methods and shows promising applications in malware analysis. In the field of cybersecurity, the application of convolutional neural networks (CNNs) has become a common practice for malware identification. CNNs excel in classifying malware with their ability to automatically extract features from raw data. However, despite their widespread use, CNNs are often criticized for their significant parameter redundancy, which requires appropriate simplification.

[0007] The main challenge facing CNNs lies in their dense architecture, which often results in too many parameters. These parameters, in turn, lead to increased computational complexity and memory requirements, making CNNs resource-intensive to train and deploy. In addition, too many parameters may also lead to overfitting, which impairs the generalization ability of the network and reduces its effectiveness in identifying new or variant malware. Considering the dynamic nature of malware, the need to simplify CNNs becomes even more obvious. Malware authors are constantly evolving their techniques, introducing new obfuscation methods and evasion strategies. A bloated CNN, due to its inherent inflexibility, may have difficulty adapting to these changes, limiting its long-term effectiveness. In addition, simplifying CNNs can also improve efficiency and accuracy. By removing redundant parameters and optimizing the network structure, we can maintain or even improve classification performance while reducing the computational burden. This not only makes the training process faster and more cost-effective, but also enables the deployment of CNNs in resource-constrained environments, such as edge computing devices.

[0008] In the field of malware classification, convolutional neural networks have become a powerful tool due to their ability to extract hierarchical features from raw data. Over the years, a large number of studies have explored various ways to improve the performance of CNNs in this field. The latest progress in CNN-based malware classification includes four main directions: (1) CNN architecture optimization, (2) CNN feature engineering and enhancement, (3) CNNs combined with hybrid and ensemble techniques, and (4) CNNs using transfer learning and domain adaptation. However, there are several limitations to current research:

[0009] 1. Although network architecture optimization and feature engineering techniques have shown promising results, they usually introduce additional computational complexity and resource requirements, limiting their applicability in resource-constrained environments or real-time scenarios.

[0010] 2. Although hybrid and ensemble methods improve the overall performance, they may suffer from increased model complexity and training time, which may hinder their practical deployment.

[0011] 3. Transfer learning and domain adaptation methods rely on the availability of relevant pre-trained models or source domains, which may not always be feasible or appropriate in specific malware detection tasks.

[0012] 4. Existing CNN-based methods mainly focus on improving classification accuracy, but ignore the redundancy and efficiency aspects of CNN architecture, which are crucial for practical deployment and real-time analysis. Summary of the invention

[0013] The purpose of the present invention is to provide a method and system for classifying malicious code images in order to reduce model redundancy when using a CNN architecture to classify malicious code images and improve the efficiency of malicious code image classification.

[0014] The technical solution adopted by the present invention is as follows:

[0015] A malicious code image classification method, comprising:

[0016] Obtain a malicious code image dataset and construct a training set and a test set, where the malicious code images in the training set are annotated with classification labels;

[0017] Using the training set to train the pre-built classification network, and using the test set to test the trained classification network to obtain a classification model;

[0018] Using the classification model to classify the malicious code image to be classified;

[0019] Wherein, the classification network is constructed as follows:

[0020] Use the Stem sub-network to perform the first convolution processing on the input malicious code image;

[0021] A multi-stage sub-network is used to perform a multi-stage second convolution process on the feature map after the first convolution process, and a transition sub-network is used between each two adjacent stages to perform a third convolution process on the convolution map obtained in the previous stage;

[0022] In the first convolution process, the second convolution process and the third convolution process, the number of input channels of each convolution kernel is configured as one, and is mapped back to the full-size convolution using the kernel generation function;

[0023] The feature map output by the last stage is classified by the classifier sub-network.

[0024] Furthermore, the Stem subnetwork performs a first convolution process on the input malicious code image using a first convolution layer and a plurality of depth-separable convolution blocks of different convolution sizes;

[0025] The depth-wise separable convolution block includes a depth-wise separable convolution layer, a batch normalization layer, and a non-linear activation function.

[0026] Furthermore, in the multi-stage sub-network, each stage includes a first depth-separable convolution layer with a first convolution kernel of size 31×31, and a second depth-separable convolution layer with a second convolution kernel of size 3×3; the first depth-separable convolution layer and the second depth-separable convolution layer are input and output in parallel during the training phase, and are merged into a single first depth-separable convolution layer input and output through structural reparameterization during the testing phase.

[0027] Furthermore, in the multi-stage sub-network, after batch normalization of the input feature map in each stage, point convolution is performed, and then batch normalization and non-linear activation are performed. For the training stage, the features are respectively input into the first depth-separable convolution layer and the second depth-separable convolution layer for convolution processing, and for the test stage, the features are input into the merged first depth-separable convolution layer for convolution processing; after batch normalization and non-linear activation, the features after convolution processing are residually connected with the input feature map.

[0028] Furthermore, the method of merging the first depthwise separable convolutional layer and the second depthwise separable convolutional layer by structural reparameterization includes:

[0029] The convolution kernels of the first depth-wise separable convolution layer and the second depth-wise separable convolution layer are respectively mapped using a kernel generation function;

[0030] Aggregate the parameters of the first depth-wise separable convolutional layer and the second depth-wise separable convolutional layer and the batch normalization layer connected thereto respectively;

[0031] Merge parameters of a first depth-wise separable convolutional layer and a second depth-wise separable convolutional layer, wherein the parameters of the second depth-wise separable convolutional layer are merged into a middle position of the parameters of the first depth-wise separable convolutional layer.

[0032] Furthermore, the method for the first depth-separable convolution layer or the second depth-separable convolution layer to perform convolution processing on the feature map X is:

[0033]

[0034] Among them, k represents the order of the output channel, (i, j) represents the horizontal and vertical coordinates of the pixel points on the feature map, and Y i,j,k Represents the convolution result of the pixel with coordinates (i, j) on the k-th output channel;

[0035] Represents the offset set of the pixel point (i, j) in the feature map in the spatial neighborhood, where u and v are the horizontal and vertical offset distances respectively, and k hand k w Respectively represent the horizontal and vertical spatial range of the convolution kernel; X i+u,j+v,k Represents the pixel with coordinates (i+u,j+v) on the k-th input channel of feature map X. represents the convolution kernel F specific to the kth input channel of the feature map X k The upper coordinate is The value of .

[0036] Furthermore, the third convolution processing is to perform a depth-separable convolution operation on the input feature map.

[0037] Furthermore, the Stem subnetwork, each stage of the multi-stage subnetwork, and each of the transition subnetworks finally randomly arrange the channels of the convolutional graphs output by themselves.

[0038] Furthermore, the classifier subnetwork performs global average pooling and global maximum pooling on the input feature map, and then fuses the two pooling features into the fully connected layer for classification.

[0039] To solve the above problem, the present invention further provides a malicious code image classification system, which is configured to execute the above malicious code image classification method.

[0040] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0041] The present invention achieves the agnosticism of the input channels by reducing the number of input channels of each convolution kernel in the network to one, and then using the kernel generating function to perform channel recovery. Compared with the traditional standard convolution, the model parameter amount of the CNN architecture is greatly reduced, especially when the number of input channels in the standard convolution is large, the reduction in the total parameter amount is more obvious, thereby improving the efficiency and performance of the network model.

[0042] The present invention utilizes a larger convolution kernel to expand the receptive field of the network, which can improve spatial specificity. In view of the challenge brought by the fact that the local changes of malicious code images are not as obvious as those of natural images, the present invention can better integrate the global information in the image, more effectively capture the subtle changes in the malicious code images, and thus more accurately identify the malicious code images. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0044] Figure 1 This is the overall architecture diagram of EssicaNet feature extraction.

[0045] Figure 2 It is the convolution flow chart of standard convolution.

[0046] Figure 3 This is the convolution flow chart in EssicaNet.

[0047] Figure 4 This is a schematic diagram of the channel shuffle operation.

[0048] Figure 5 This is a schematic diagram of the channel random arrangement operation.

[0049] Figure 6 This is the training and testing flow chart of EssicaNet.

[0050] Figure 7 These are the training curves of different models on the MalImg dataset, including: (a) the recognition accuracy change curve of different models on the training set; (b) the cross entropy loss function change curve of different models on the training set; (c) the recognition accuracy change curve of different models on the test set; (d) the cross entropy loss function change curve of different models on the test set.

[0051] Figure 8 These are the training curves of different models on the BIG2015 dataset, including: (a) the recognition accuracy change curve of different models on the training set; (b) the cross entropy loss function change curve of different models on the training set; (c) the recognition accuracy change curve of different models on the test set; (d) the cross entropy loss function change curve of different models on the test set. DETAILED DESCRIPTION

[0052] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.

[0053] Any feature disclosed in this specification (including any additional claims and abstract), unless otherwise stated, may be replaced by other equivalent or alternative features having similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0054] The malicious code image classification method includes the following processes:

[0055] First, we need to build a malicious code image dataset for training the classification network. The dataset contains a large number of malicious code image samples collected, and these malicious code images as samples are labeled with classification labels, that is, which category of malicious code these malicious code images belong to. On this basis, a certain proportion of samples are extracted from the dataset to construct the training set and test set.

[0056] The pre-built classification network is trained using the training set (samples), and the trained classification network is tested using the test set (samples) to obtain a classification model.

[0057] After the training is completed, the classification model is used to classify the malicious code images to be classified.

[0058] The above-mentioned pre-built classification network is referred to as EssicaNet in this embodiment, where Essica stands for Enhanced Spatial Specificity and Input Channel Agnosticism, i.e. enhanced spatial specificity and input channel independence.

[0059] like Figure 1 As shown in Figure 2, the feature extraction part of EssicaNet mainly consists of three parts: the Stem sub-network, the multi-stage sub-network, and the transition sub-network (Transition) that connects each stage of the multi-stage sub-network.

[0060] EssicaNet uses the Stem subnetwork to perform the first convolution processing on the input malicious code image; uses the multi-stage subnetwork to perform the multi-stage second convolution processing on the feature map after the first convolution processing, and uses the transition subnetwork to perform the third convolution processing on the convolution map obtained in the previous stage between each two adjacent stages. The feature map output in the last stage is classified by the classifier subnetwork.

[0061] The Stem subnetwork is intuitive and important in the initial stage of EssicaNet operation, which uses the first convolution layer and multiple depth-separable convolution blocks of different convolution sizes to perform the first convolution on the input malicious code image. Specifically, the Stem subnetwork captures the low-level details and patterns of the input malicious code image through a series of EssicaConv layers (i.e., the first convolution layer) / EssicaDWConv blocks (i.e., depth-separable convolution blocks).

[0062] In some embodiments, the Stem subnetwork uses a 3×3 EssicaConv layer and multiple EssicaDWConv blocks with different convolution sizes. Each EssicaDWConv block contains an EssicaDWConv layer, i.e., a depthwise separable convolution layer. The convolution kernel size of the EssicaDWConv layer here is also assumed to be 3×3. The EssicaDWConv layer is followed by a batch normalization layer and a nonlinear activation layer. The nonlinear activation layer is configured with a nonlinear activation function. In this embodiment, the nonlinear activation function uses the Mish function.

[0063] The multi-stage sub-network is the core component of EssicaNet, which consists of multiple EssicaRepLK (re-parameterized large core EssicaDWConv) blocks. Each EssicaRepLK block is equivalent to a stage of the multi-stage sub-network, such as Figure 1 As shown, it is assumed that the multi-stage sub-network contains three extremes, the first stage Stage1 contains B1 EssicaRepLK blocks, the second extreme Stage2 contains B2 EssicaRepLK blocks, and the third stage Stage3 contains B3 EssicaRepLK blocks.

[0064] Each EssicaRepLK block contains the first depthwise separable convolutional layer with the first convolution kernel of size 31×31, i.e. Figure 1 The large EssicaDWConv in also includes a second depth-wise separable convolutional layer with a second convolutional kernel of size 3×3, i.e. Figure 1 In the training phase, large EssicaDWConv and Small EssicaDWConv are input and output in parallel, while in the testing phase, large EssicaDWConv and Small EssicaDWConv are merged into a single large EssicaDWConv input and output through structural reparameterization (StructRep).

[0065] Specifically, if Figure 1 As shown, each EssicaRepLK block sequentially includes a batch normalization layer, a point-wise convolution (PWConv) layer, a batch normalization layer, a nonlinear activation layer, and the above-mentioned parallel large EssicaDWConv layer and Small EssicaDWConv layer, and the large EssicaDWConv layer and the Small EssicaDWConv layer are followed by a batch normalization layer, respectively. After that, the EssicaRepLK block also includes a nonlinear activation layer, and the EssicaRepLK block is also residually connected with the input feature map after the nonlinear activation layer. That is, in each stage, after batch normalization, point-wise convolution is performed on the input feature map, and then batch normalization and nonlinear activation are performed. For the training stage, the first depth-separable convolution layer and the second depth-separable convolution layer are respectively input for convolution processing, and for the test stage, the first depth-separable convolution layer after the merger is input for convolution processing; the convolution-processed features are batch normalized and nonlinearly activated, and then residually connected with the input feature map.

[0066] The transition subnetwork is located between adjacent EssicaRepLK blocks to promote information propagation at different scales and help capture multi-scale representations. In some embodiments, the third convolution process of the transition subnetwork is to perform a depth-separable convolution operation on the input feature map, where the convolution kernel can also be designed to be 3×3.

[0067] As a major point of the present invention, the convolution of EssicaNet is specially designed to overcome the problem of network parameter redundancy caused by traditional standard convolution.

[0068] Given an input feature map C in represents the number of input channels of the input feature map, H and W represent the horizontal and vertical dimensions of each input channel respectively. represents the output feature map of the standard convolution, C out represents the number of output channels of the output feature map, and Respectively represent the horizontal and vertical dimensions of each output channel, then:

[0069]

[0070] Among them, c represents the order of input channels, k represents the order of output channels, (i, j) represents the horizontal and vertical coordinates of the pixel points on the input feature map, and Y i,, Represents the convolution result of the pixel with coordinates (i, j) on the k-th output channel; Represents the offset set of the pixel point (i, j) in the input feature map in the spatial neighborhood, where u and v are the horizontal and vertical offset distances respectively, and k h and k w Respectively represent the horizontal and vertical spatial range of the convolution kernel; X i+u,, Represents the pixel with coordinates (i+u,j+v) on the cth input channel of feature map X. represents the convolution kernel W specific to the cth input channel and kth output channel of the input feature map X c,k The upper coordinate is The convolution process of standard convolution is as follows Figure 2 shown.

[0071] In order to solve the problem of input channel redundancy in the standard convolution analyzed in the background technology part, the convolution operation in EssicaNET is specially designed in this embodiment. Figure 3 As shown, the number of input channels of the convolution kernel in the standard convolution is reduced to one, and a new convolution kernel is obtained. Realize the agnosticism of the input channel; then, define a kernel generating function To map the reduced-dimensional convolution kernel back to the full-size convolution, the expression is

[0072] When designing a convolution operation, there are two main goals: 1) to generate a function for the kernel Define a design space; 2) Rapidly prototype a series of effective design examples to meet the needs of practical applications. In this embodiment, the convolution kernel H is selected from Expand to Therefore, the kernel generating function The dimensionality of the input channels is effectively restored, allowing the computational efficiency improvements brought by the reduced convolution kernel size to be exploited while retaining specific channel information.

[0073] The kernel generating function mentioned above With a trainable parameter G, its expression is:

[0074]

[0075] in, ⊙ denotes a multiplication operation applied only on the input channel dimension.

[0076] The number of parameters of a standard convolution is given by C in ×C out ×k h ×k w The number of parameters of the convolution operation optimized in this embodiment is determined by C in +C out ×k h ×k w The number of convolution parameters is not directly proportional to the number of input channels, but is determined by C in The optimization of the convolution kernel in this embodiment potentially reduces the total number of parameters of EssicaNet, especially in the number of input channels C. in The reduction in the total number of parameters is particularly significant when is large.

[0077] In addition, in order to improve the spatial specificity of the convolution operation, in each EssicaRepLK block of the multi-stage sub-network of this embodiment, a parallel large EssicaDWConv layer and a Small EssicaDWConv layer are introduced. On the one hand, the large EssicaDWConv with a large convolution kernel can expand the receptive field of feature extraction. On the other hand, structural reparameterization can be applied to merge the parallel large EssicaDWConv layer and the Small EssicaDWConv layer into a unified large convolution kernel DWConv (depth-separable convolution) layer. The so-called DWConv, as introduced in the paper "Xception: Deep Learning with Depthwise Separable Convolutions", maximizes the number of group convolutions.

[0078] The EssicaRepLK block first uses point-wise convolution to transform the channel dimension of the input feature map from C in Convert to C out . Then, the large EssicaDWConv layer and the Small EssicaDWConv layer are used in parallel. In this case, each EssicaDWConv layer is actually simplified to a single convolution kernel k=1,2,…,C out , independently convolving a single feature channel indexed by k. Therefore, assuming that the number of output channels is equal to the number of input channels, from H k Remove the first dimension to form F k The method by which the large EssicaDWConv layer and the Small EssicaDWConv layer perform convolution processing on the feature map X is expressed as:

[0079]

[0080] represents the convolution kernel F specific to the kth input channel of the feature map X k The upper coordinate is The value of .

[0081] It should be noted that the convolution kernel F k The kth feature slice X specific to the input feature map in the channel dimension ·,·,k , and is shared across all spatial locations in this slice. This specificity allows for more focused and efficient convolution operations that adapt to the unique characteristics of each feature channel.

[0082] During the testing phase, the structural reparameterization StructRep includes the following three steps:

[0083] 1) The large EssicaDWConv layer and the Small EssicaDWConv layer are aggregated with their parameters G in EssicaNet using formula (2). That is, the kernel generating function The convolution kernels of the large EssicaDWConv layer and the SmallEssicaDWConv layer are mapped respectively.

[0084] 2) Based on step 1), the large EssicaDWConv layer and the Small EssicaDWConv layer are respectively aggregated with the parameters of the batch normalization layer that follows them in the EssicaRepLK block. This step ensures that the statistical properties of the activations are effectively integrated into the EssicaDWConv layer, allowing for a more comprehensive representation of the learned features.

[0085] 3) In the aggregation step 2), the parameters of the aggregated large EssicaDWConv layer and the Small EssicaDWConv layer are merged, where the parameters of the Small EssicaDWConv layer are added to the middle of the parameters of the large EssicaDWConv layer. This integration process produces a unified set of parameters that encompasses the combined power of the original different EssicaDWConv filters. Therefore, the structural reparameterization simplifies the network by reducing the number of different layers while retaining the essential properties necessary for accurate feature extraction and representation.

[0086] StructRep is typically executed as an operation after EssicaNet training is complete, with the goal of reducing computational overhead during the testing phase. This strategic post-training adjustment allows the network architecture to be optimized by integrating or reconfiguring certain parameters and layers to minimize computational cost without sacrificing the accuracy and robustness of the model's predictions.

[0087] In addition, as a preferred design, the Stem subnetwork, each EssicaRepLK block, and each transition subnetwork finally perform random channel permutations on the convolutional graphs they output. Random channel permutations are different from random channel shuffles (RandomChannel Shuffle), which are more random and thus have a better effect on enhancing scale information.

[0088] Channel shuffling is a technique to enhance information exchange between channels within a convolutional layer by grouping and redistributing the output channels to integrate information while maintaining the integrity of the feature map. Figure 4 As shown, the channel shuffling technique includes an initial step of dividing the output channels into N groups (such as Figure 4As shown in , N=3 in the figure). Then, these channels are arranged into a matrix whose number of rows corresponds to the number of groups N. Next, the matrix is ​​transposed to rearrange its rows and columns. Finally, the transposed matrix is ​​reconstructed into a one-dimensional array, thereby achieving the reordering of the output channels. It should be noted that this reordering is performed according to a specific pattern, indicating that the arrangement is not completely random, but follows a regular, albeit disrupted, sequence. In order to completely disrupt the order of channels to achieve more thorough information interaction, this embodiment proposes a channel random permutation (RandomPermutation) operation. Unlike channel shuffling, random permutation (such as Figure 5 The fully random channel rearrangement ensures extensive information exchange.

[0089] Specifically, if Figure 1 As shown in the figure, a RandomPermutation layer is designed to perform channel random permutation after the EssicaConv layer of the Stem subnetwork and after each EssicaDWConv block, after each transition subnetwork, and after the last nonlinear activation layer of each EssicaRepLK block.

[0090] After the final feature map is extracted by the Stem sub-network, the multi-stage sub-network, and the transition sub-network, EssicaNet classifies the feature map output by the last stage by the classifier sub-network.

[0091] Traditional classifier subnetworks usually use global average pooling after the convolution layer to obtain spatial feature representation, which is then input into the fully connected layer for classification. However, this embodiment also integrates and optimizes the classifier subnetwork.

[0092] In this embodiment, the classifier subnetwork performs global average pooling and global maximum pooling on the input feature map, and then fuses the two pooling features into the fully connected layer for classification.

[0093] Specifically, Represents the convolutional graph output from the last stage, H * and W * Respectively represent the horizontal and vertical sizes of the feature map.

[0094] The global average pooling operation is defined as:

[0095]

[0096] in, is the spatial feature representation obtained by global average pooling, X * i,j Represents the pixel at coordinate (i, j) in the feature map.

[0097] Similarly, the global max pooling operation is defined as:

[0098] f max =max i,j X * i,j (5)

[0099] in, It is the spatial feature representation obtained by global maximum pooling.

[0100] The final classification output is obtained by applying the fusion to f avg and f max The output of the fully connected layer is obtained:

[0101] y=FC(f avg +f max ) (6)

[0102] Where FC represents the fully connected layer and y is the final classification output.

[0103] By fusing the classification outputs from global average pooling and global max pooling, our model is able to effectively capture the spatial average features and maximum activation features, which may enhance the generalization ability and robustness of the model in malicious image classification.

[0104] The training of EssicaNet is the process of updating the network parameters of EssicaNet using the training samples of the training set. This first requires the construction of a loss function to indicate the difference between the output of EssicaNet and the true label. Let D train and D test Respectively represent the inclusion of N train The training set contains N samples and test The test set is a set of samples. Among them, (x,y)∈D train represents the input features of a sample in the training set and its corresponding true label, y = 1, 2, ..., Q, Q represents the total number of target categories. Then, in D train The cross entropy loss function on can be constructed as:

[0105]

[0106] Among them, θ represents all trainable parameters, P(y=q|x;θ) is the qth output of the Softmax classifier, which represents the probability that the input feature x is predicted to be the qth class.

[0107] Formula (7) is called the empirical risk loss function. The training of a neural network is essentially to minimize the empirical risk loss function, that is:

[0108] θ * =argminθ J(θ;D train ) (8)

[0109] For a larger N train Due to the limitations of computer memory and computing power, it is impossible to input all samples into the network at the same time to update the parameters. Therefore, a small batch SGD algorithm is usually used to randomly select B samples each time to update the network parameters. Therefore, the entire training set needs to be randomly divided into Small batches, i.e. In the tth (t∈{1,2,…,T}) mini-batch dataset The cross entropy loss function on is:

[0110]

[0111] T small batch samples are input into EssicaNet for training in sequence and updated using the SGD algorithm. In the tth iteration, the parameters are updated as follows:

[0112] θ t+1 =θ t -αg t (10)

[0113] Where α is the learning rate, g t is the loss function with respect to the parameter θ t The gradient of is:

[0114]

[0115] The prediction process of sample x is to maximize the posterior probability, that is:

[0116]

[0117] in, is the predicted label.

[0118] like Figure 6 As shown in the figure, the training and testing process of EssicaNet includes the following parts:

[0119] 1) Model construction: Based on the above description of EssicaNet, the EssicaNet+Softmax classifier structure is constructed;

[0120] 2) Loss function construction: construct the cross-entropy loss function (Cross-Entropy Loss), as shown in formula (9);

[0121] 3) Model training: EssicaNet is trained using the training set, and model training is completed through multiple rounds of forward propagation and back propagation. The forward propagation formula is formula (9), the back propagation uses formula (11), and the model update uses formula (10);

[0122] 4) Structural reparameterization: After model training is completed, structural reparameterization is used to merge parameters to ensure efficient prediction;

[0123] 5) Model testing: Input the malicious code image to be tested into the trained EssicaNet and use formula (12) to predict the result.

[0124] Example 2

[0125] This embodiment proposes a malicious code image classification system, which is configured to execute the malicious code image classification method of embodiment 1.

[0126] Specifically, the system is configured with a processor and a memory, the memory stores a computer program, and running the computer program can execute the method of Embodiment 1, and the processor is configured to execute the computer program in the memory.

[0127] Example 3

[0128] This embodiment uses two malicious code image datasets, MalImg and BIG2015, to conduct classification experiments, and selects 18 different network models (including EssicaNet and its variants, and some other networks that perform well in image classification tasks) for comparative experiments to verify the performance of the method of Embodiment 1. MalImg and BIG2015 contain image samples of various malicious codes.

[0129] The experiment used the PyTorch2.3.0 deep learning open source framework and Python3.12 programming language to build the network model. All models used the same training strategy and parameter settings to ensure fair comparison. The specific training strategy and parameter settings are as follows: the optimizer used for training is RAdam, the learning rate is 0.001, the total number of training cycles is 200, and the learning rate is decayed at the 60th, 120th and 180th cycles, that is, the learning rate is multiplied by 0.1 at these cycles respectively. During the training process, all images are resized to 64×64, and the cross entropy loss function is used as the objective function of training. The batch size is set to 64.

[0130] This embodiment uses the BIG dataset and the MalImg dataset. The BIG dataset comes from the 2015 Microsoft Malware Classification Challenge, and contains 9 malicious code families, a total of 10,860 samples, of which the number of training set samples is 8,684 and the number of test set samples is 2,176. The MalImg dataset is an image database containing malware samples. The dataset consists of 25 different malware families, a total of 9,458 samples, of which the number of training set samples is 8,404 and the number of test set samples is 935. It is derived from the paper "Malware Images: Visualization and Automatic Classification". Each sample is converted into a grayscale image for visualization and classification using image processing technology. This conversion is based on the observed fact that for many malware families, samples belonging to the same family show significant visual similarities in layout and texture. Therefore, a method of classification by image features is proposed, without the need for disassembly or code execution, thereby simplifying the classification process and improving efficiency.

[0131] Table 1 Datasets used in the experiments

[0132]

[0133]

[0134] In this example, the proposed EssicaNet is combined with a series of advanced deep learning models that have performed well in image classification tasks, especially in the field of malware detection. These models include but are not limited to the following:

[0135] 1) ConvNeXtV2_Tiny: As a lightweight version of the ConvNeXt series, ConvNeXtV2_Tiny achieves the goal of significantly reducing model parameters and computational complexity while maintaining high accuracy by introducing a more efficient network structure and optimization strategy. The model demonstrates excellent performance in image classification tasks and is particularly suitable for resource-constrained environments.

[0136] 2) FastViT_s12: FastViT_s12 is a lightweight model based on Vision Transformer, which achieves fast image processing capabilities by reducing the computational effort of the attention mechanism. This model significantly improves processing speed while maintaining high accuracy, making it particularly suitable for scenarios that require real-time response.

[0137] 3) GhostNetV2_100: GhostNetV2_100 is an upgraded version of the GhostNet series. By introducing a more efficient feature extraction mechanism, it achieves the goal of significantly reducing model parameters and computational complexity while maintaining high accuracy. The model has demonstrated excellent performance in image classification tasks and is particularly suitable for resource-constrained environments.

[0138] 4) Inception_Next_Small: Inception_Next_Small is a lightweight version of the Inception series. By introducing a more efficient network structure and optimization strategy, it achieves the goal of significantly reducing model parameters and computational complexity while maintaining high accuracy. This model has demonstrated excellent performance in image classification tasks and is particularly suitable for resource-constrained environments.

[0139] 5) MalMKNet: MalMKNet is an innovative deep learning model designed for fast and accurate identification of unknown malicious code and its variants. It is derived from the paper "MalMKNet: A Multi-Scale Convolutional Neural Network for Malware Classification". The model significantly improves the recognition ability of malicious code by converting malicious code into grayscale images and using a CNN architecture with a mixture of multi-scale convolution kernels. MalMKNet not only improves the accuracy of the model but also effectively reduces the number of model parameters by combining a hybrid convolution kernel module that combines deep large kernel convolution and standard small kernel convolution, as well as multi-scale kernel fusion technology.

[0140] 6) MnasNet_050: MnasNet_050 is a lightweight model based on Neural Architecture Search (NAS) technology. By automatically searching for the optimal network structure, it achieves the goal of significantly reducing model parameters and computational complexity while maintaining high accuracy. The model has demonstrated excellent performance in image classification tasks and is particularly suitable for resource-constrained environments.

[0141] 7) RepVGG_A0: RepVGG_A0 is a Vision Transformer model based on a repetitive structure. By reusing the same convolutional blocks, it achieves the goal of significantly reducing model parameters and computational complexity while maintaining high accuracy. This model has shown excellent performance in image classification tasks and is particularly suitable for resource-constrained environments.

[0142] 8) RepViT_M1: RepViT_M1 is a Vision Transformer model based on a repetitive structure. By reusing the same transformation blocks, it achieves the goal of significantly reducing model parameters and computational complexity while maintaining high accuracy. This model has shown excellent performance in image classification tasks and is particularly suitable for resource-constrained environments.

[0143] 9) ResNet18: ResNet18 is a classic residual network model that solves the gradient vanishing problem in deep network training by introducing residual connections. This model has shown good performance in image classification tasks and has been widely used in various visual tasks due to its simple structure.

[0144] 10) SE ResNet18: SE ResNet18 is a model based on residual network. By introducing SE (Squeeze-and-Excitation) module, the model's attention mechanism on feature channels is enhanced. This model shows higher accuracy and stronger feature expression ability in image classification tasks.

[0145] 11)SEResNeXt101_32x8d,SEResNeXt50_32x4d: These SEResNeXt models are deep networks based on residual networks and grouped convolutions. By introducing grouped convolutions and SE modules, they achieve the goal of significantly reducing model parameters and computational complexity while maintaining high accuracy. These models show excellent performance in image classification tasks and are particularly suitable for scenarios that require high accuracy and high efficiency.

[0146] 12) SKResNet18: SKResNet18 is a model based on residual network. By introducing the SK (Selective Kernel) module, it can dynamically adjust the importance of features of different scales. This model shows higher accuracy and stronger feature expression ability in image classification tasks.

[0147] 13) Xception41: Xception41 is a model based on depthwise separable convolution, which significantly improves the efficiency of the model by decomposing the traditional convolution operation into depthwise convolution and pointwise convolution. The model has shown excellent performance in image classification tasks and has been widely used in various visual tasks due to its efficient computing power.

[0148] These models show different advantages and characteristics in the malware image classification task, providing diverse options for malware detection. By comprehensively considering the performance, efficiency, and resource requirements of the model, the most suitable model can be selected for different application scenarios to achieve efficient and accurate malware detection.

[0149] Table 2 is the experimental results of the proposed EssicaNet on the MalImg dataset compared with other network models. As can be seen from Table 2, the EssicaNet series models perform well in terms of loss and accuracy in the training phase. Among them, the EssicaNet31s3, EssicaNet31s5, EssicaNet31s7 and EssicaNet31s9 models have an accuracy of 100% on the training set and a loss of 0, which shows that the model can perfectly fit the data during training. On the validation set, the EssicaNet series models also performed well, with the highest accuracy reaching 99.67% and the lowest validation loss being 0.03, which is much lower than other models. In particular, the accuracy of the EssicaNet31s3, EssicaNet31s5, EssicaNet31s7 and EssicaNet31s9 models on the validation set exceeded 99%, which shows that EssicaNet has a significant advantage in generalization ability. In addition, the number of parameters of the EssicaNet series models is much lower than that of other models. For example, the number of parameters of EssicaNet31s3 is only 126586, while the number of parameters of other models such as ResNet18 and SEResNet18 are 11183065 and 11273177 respectively. This shows that EssicaNet has achieved lightweight model while maintaining high accuracy.

[0150] Table 3 shows the experimental results of the proposed EssicaNet on the BIG2015 dataset compared with other network models. As can be seen from Table 3, the EssicaNet series models perform very well on the training set. All EssicaNet models achieve 100% training accuracy and a training loss of 0, which shows that the model can perfectly learn the features in the dataset. On the validation set, the EssicaNet series models also perform well, with the highest validation accuracy reaching 98.667% and the lowest validation loss being 0.125, which shows that the EssicaNet model has a significant advantage in generalization ability. In particular, the EssicaNet31s3, EssicaNet31s5, EssicaNet31s7, and EssicaNet31s9 models all have an accuracy of more than 98% on the validation set, which is very important in the malicious code detection task, because high accuracy means that the model can more effectively identify malicious code, thereby improving the security of the system. In addition, the number of parameters of the EssicaNet series models is much lower than that of other models. For example, the number of parameters of EssicaNet31s3 is only 124522, while the number of parameters of other models such as ResNet18 and SeresNet18 are 11174857 and 11264969 respectively. This shows that EssicaNet has achieved lightweight model while maintaining high accuracy.

[0151] Table 2 Comparative experimental results of EssicaNet and other network models on the MalImg dataset

[0152]

[0153]

[0154] Table 3 Comparative experimental results of EssicaNet and other network models on the BIG2015 dataset

[0155]

[0156] In summary, the EssicaNet series of models have demonstrated excellent performance in the malicious code image classification task. Through the carefully designed network structure and training strategy, EssicaNet not only achieves high accuracy and low loss on the training set, but also shows excellent generalization ability and robustness on the validation set. In addition, the lightweight design of EssicaNet makes it more efficient and has lower computational cost in practical applications.

[0157] Figure 7 and Figure 8 They are the training curves of different models on the MalImg and BIG2015 datasets. Figure 7 and Figure 8 It can be seen that EssicaNet achieved 100% recognition accuracy and a loss function value of 0 in both training data sets, proving the effectiveness of EssicaNet. In the test set, EssicaNet has a more obvious performance advantage over the comparison method, with a higher training set recognition accuracy asymptote and a lower loss function asymptote, highlighting the advanced nature of the method of the present invention.

[0158] The present invention is not limited to the above-mentioned specific embodiments, but extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.

Claims

1. A malicious code image classification method, characterized in that: include: Obtain a malicious code image dataset and construct a training set and a test set, where the malicious code images in the training set are annotated with classification labels; Using the training set to train the pre-built classification network, and using the test set to test the trained classification network to obtain a classification model; Using the classification model to classify the malicious code image to be classified; Wherein, the classification network is constructed as follows: Use the Stem sub-network to perform the first convolution processing on the input malicious code image; A multi-stage subnetwork is used to perform a multi-stage second convolution processing on the feature map after the first convolution processing, and a transition subnetwork is used between each two adjacent stages to perform a third convolution processing on the convolution map obtained in the previous stage; in the first convolution processing, the second convolution processing and the third convolution processing, the number of input channels of each convolution kernel is configured as one, and the kernel generation function is used to map back to the full-size convolution; The feature map output by the last stage is classified by the classifier sub-network.

2. The malicious code image classification method according to claim 1, characterized in that: The Stem sub-network performs a first convolution process on the input malicious code image using a first convolution layer and a plurality of depth-separable convolution blocks of different convolution sizes; The depth-wise separable convolution block includes a depth-wise separable convolution layer, a batch normalization layer, and a non-linear activation function.

3. The malicious code image classification method according to claim 1, characterized in that: In the multi-stage sub-network, each stage includes a first depth-wise separable convolution layer with a first convolution kernel of size 31×31, and a second depth-wise separable convolution layer with a second convolution kernel of size 3×3; the first depth-wise separable convolution layer and the second depth-wise separable convolution layer are input and output in parallel during the training phase, and are merged into a single first depth-wise separable convolution layer input and output through structural reparameterization during the testing phase.

4. The malicious code image classification method according to claim 3, characterized in that: In the multi-stage sub-network, after batch normalization is performed on the input feature map in each stage, point convolution is performed, and then batch normalization and non-linear activation are performed. For the training stage, the features are respectively input into the first depth-wise separable convolution layer and the second depth-wise separable convolution layer for convolution processing. For the test stage, the features are input into the merged first depth-wise separable convolution layer for convolution processing. After batch normalization and non-linear activation, the features after convolution processing are residually connected with the input feature map.

5. The malicious code image classification method according to claim 4, characterized in that: The method of merging the first depth-wise separable convolutional layer and the second depth-wise separable convolutional layer by structural reparameterization includes: The convolution kernels of the first depth-wise separable convolution layer and the second depth-wise separable convolution layer are respectively mapped using a kernel generation function; Aggregate the parameters of the first depth-wise separable convolutional layer and the second depth-wise separable convolutional layer and the batch normalization layer connected thereto respectively; Merge parameters of a first depth-wise separable convolutional layer and a second depth-wise separable convolutional layer, wherein the parameters of the second depth-wise separable convolutional layer are merged into a middle position of the parameters of the first depth-wise separable convolutional layer.

6. The malicious code image classification method according to claim 4, characterized in that: The method of performing convolution processing on the feature map X by the first depth-separable convolution layer or the second depth-separable convolution layer is: Among them, k represents the order of the output channel, (i, j) represents the horizontal and vertical coordinates of the pixel points on the feature map, and Y i,j,k Represents the convolution result of the pixel with coordinates (i, j) on the k-th output channel; Represents the offset set of the pixel point (i, j) in the feature map in the spatial neighborhood, where u and v are the horizontal and vertical offset distances respectively, and k h and k w Respectively represent the horizontal and vertical spatial range of the convolution kernel; X i+u,j+v,k Represents the pixel with coordinates (i+u,j+v) on the k-th input channel of feature map X. represents the convolution kernel F specific to the kth input channel of the feature map X k The upper coordinate is The value of .

7. The malicious code image classification method according to claim 2, characterized in that: The third convolution process is to perform a depth-separable convolution operation on the input feature map.

8. The malicious code image classification method according to any one of claims 1 to 7, characterized in that: The Stem subnetwork, each stage of the multi-stage subnetwork, and each of the transition subnetworks finally perform random channel arrangement on the convolution graph output by itself.

9. The malicious code image classification method according to claim 1, characterized in that: The classifier subnetwork performs global average pooling and global maximum pooling on the input feature map, and then fuses the two pooling features into the fully connected layer for classification.

10. A malicious code image classification system, characterized in that: The system is configured to execute the malicious code image classification method as described in any one of claims 1 to 9.

Citation Information

Patent Citations

  • Training and detecting method and device of malicious code family

    CN107392019A

  • Malicious software detection method and device based on deep learning

    CN110765458A

  • Malicious code classification method and device, electronic equipment and medium

    CN114741697A

  • Malicious code classification method and device, storage medium and electronic equipment

    CN117521067A

  • Remote sensing image road extraction method, system and equipment based on large kernel convolution and direction stripe convolution, and medium

    CN117789028A