Malicious software detection method based on class distance optimization

Abstract

The invention provides a malicious software detection method based on class distance optimization, and the method comprises the steps: obtaining malicious software sample data, and carrying out the preprocessing of the malicious software sample data; decompiling the preprocessed data to obtain a source code file; selecting byte code features from the source code file, and converting the byte code features into an RGB three-channel image; the RGB three-channel image is expanded by adopting the optimized GAN architecture, and an expanded malicious software sample is obtained; inputting the expanded malicious software sample into a deep neural network to obtain a malicious software family classification result; the generated high-quality malicious software image can be used for data enhancement, so that the model is more stable when facing various malicious software variants, the over-fitting risk is reduced, the reliability of a detection system is improved, the data acquisition cost is reduced, a data set is over-expanded, and the detection accuracy is improved. The deep learning model covers more possible malicious software features in the training process, so that manual intervention is reduced, and the automation level of a detection system is improved.

Description

Technical Field

[0001] This invention belongs to the field of malware detection in network security, and specifically relates to a malware detection method based on class-based optimization. Background Technology

[0002] Malware refers to a set of programs or code that are deliberately designed and compiled to access or damage computer systems or networks without authorization, or to steal or tamper with data, ultimately for the purpose of illegal profit or functional disruption.

[0003] With their portability, abundant sensors, and diverse functions, smartphones have become an integral part of daily life, deeply integrated into social networking, entertainment, payments, photography, and identity verification. However, as the amount of private information stored on these devices—such as photos, contacts, and account passwords—has increased, security issues have become increasingly prominent. Android, which dominates the global smart mobile device market, makes its security research particularly crucial. While providing convenience, the mobile internet has also brought serious security threats such as malware, making it a key focus in the current security field.

[0004] In recent years, malware visualization technology has been widely adopted in the field of malware detection, converting malware samples into images for deep learning analysis. However, current malware datasets generally suffer from class imbalance, resulting in significant differences in the number of images in different classes after conversion, affecting the classification performance of the model. During training, deep learning models tend to favor majority class samples, exhibiting weaker ability to identify minority class malware, leading to decreased detection accuracy, especially when facing novel, rare, or highly variant malware, which is prone to false positives or false negatives. Furthermore, class imbalance also affects the model's generalization ability, reducing its adaptability to real-world environments and thus increasing cybersecurity risks. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes a malware detection method based on class distance optimization. The method includes: converting the bytecode of the malware to be detected into an RGB three-channel image; mapping the RGB three-channel image to obtain a high-quality synthetic sample; and inputting the high-quality synthetic sample into a pre-trained malware detection model to obtain the detection result.

[0006] Pre-training the malware detection model includes: acquiring malware sample data and preprocessing the malware sample data; decompiling the preprocessed data to obtain source code files; selecting bytecode features from the source code files and converting the bytecode features into RGB three-channel images; augmenting the RGB three-channel images using an optimized GAN architecture to obtain augmented malware samples; inputting the augmented malware samples into a deep neural network to obtain malware family classification results; calculating the model's loss function based on the malware family classification results, adjusting the model's parameters, and completing the model training when the loss function converges.

[0007] The beneficial effects of this invention are:

[0008] This invention proposes a malware detection model enhancement method based on class distance optimization and adversarial sample generation. This method involves decompiling malware and selecting the file that best reflects the malware's behavioral characteristics. This file is then converted into an RGB image using a proposed visualization algorithm. Subsequently, a fine-tuned GAN framework is used, and a hybrid loss function is employed to adjust the distribution distance between the same and different classes to generate high-quality malware images. Generating high-quality malware images increases the number of samples from rare classes in the training set, preventing the classifier from being biased towards common classes during training and improving overall detection performance. Due to the rapid evolution of malware, traditional detection methods perform poorly when faced with unseen samples. By improving the samples generated by GAN, the characteristics of novel malware in the real world can be better simulated, thereby enhancing the model's generalization ability to unknown malware. The generated high-quality malware images can be used for data augmentation, making the model more robust to various malware variants, reducing the risk of overfitting, improving the reliability of the detection system, and lowering data acquisition costs. Over-expanding the dataset allows the deep learning model to cover more possible malware features during training, thereby reducing human intervention and improving the automation level of the detection system. Attached Figure Description

[0009] Figure 1 This is a structural diagram of the model framework of the present invention;

[0010] Figure 2 This is a schematic diagram illustrating the inter-class and intra-class distances of the present invention;

[0011] Figure 3 This is the overall flowchart of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] A malware detection method based on class range optimization, such as Figures 1-3 As shown, the method includes: converting the bytecode of the malware to be detected into an RGB three-channel image; mapping the RGB three-channel image to obtain a high-quality synthetic sample; and inputting the high-quality synthetic sample into a pre-trained malware detection model to obtain the detection result.

[0014] Pre-training the malware detection model includes: acquiring malware sample data and preprocessing the malware sample data; decompiling the preprocessed data to obtain source code files; selecting bytecode features from the source code files and converting the bytecode features into RGB three-channel images; augmenting the RGB three-channel images using an optimized GAN architecture to obtain augmented malware samples; inputting the augmented malware samples into a deep neural network to obtain malware family classification results; calculating the model's loss function based on the malware family classification results, adjusting the model's parameters, and completing the model training when the loss function converges.

[0015] In this embodiment, the present method proposes an enhancement method for malware detection models based on class-range optimized adversarial sample generation, such as... Figure 1 As shown, the proposed method includes the following steps: acquiring malware samples and decompiling them to obtain source code, then extracting bytecode or opcode features from the source code. The extracted features are then converted into images, resulting in an image dataset categorized according to malware family types. This dataset is then input into a fine-tuned GAN network to obtain high-quality malware image dataset samples, ensuring consistency across each category and effectively addressing the imbalance problem in malware datasets.

[0016] Methods to enhance malware detection models based on class-range optimized adversarial example generation include:

[0017] S1: Obtain the malware dataset and decompile it to obtain the bytecode or opcode file of the dataset.

[0018] Obtain malware samples and extract files from multiple categories; extract 20,000 malware samples containing 9 categories from the Microsoft BIG dataset, and extract the bytecode and opcode files from the malware.

[0019] S2: Convert the obtained malware sample into an RGB image. The specific implementation process is as follows:

[0020] First, the input malware file is read line by line, split into atomic tokens by spaces, allowing only hexadecimal numbers (0-9, af, AF) and the '?' symbol. Invalid characters are forcibly replaced with '00', preserving the original data topology. Then, '?' is mapped to 0x00, and other hexadecimal values ​​are converted to integers (0-255). Modulo operations are used to dynamically pad with zeros, ensuring the byte stream length is a multiple of 3, generating a standard RGB triplet sequence: each element corresponds to a (R, G, B) channel value. A minimum enclosing strategy is used to determine the image size, and finally, floating-point normalization is performed: the integer value range [0, 255] is linearly mapped to [0, 1]. Explicit data type conversion is then performed: a float32 tensor conforming to the deep learning framework standard is generated. Finally, a three-channel RGB image is obtained. S3: The obtained image dataset is input into the fine-tuned GAN framework.

[0021] Conventional GAN ​​frameworks cannot handle multi-class data. Therefore, based on the idea of ​​ACGAN, class labels are added to the generator model, using class information as a condition for image generation, thus generating images of specific classes. In addition, it can learn latent spatial information unrelated to class labels. However, conventional ACGAN models may face certain problems when processing complex malware images, resulting in low generation quality. Therefore, this framework has been improved. Figure 1 As shown, the malware image dataset obtained through step S2 is input into the adjusted GAN framework.

[0022] This framework assigns each sample a corresponding class number c and noise Z, and the generator uses these two pieces of information to generate... In this way, the objective function of the model is derived from the source class. and correct class The likelihood function is constructed. Its formulas are shown in (3) and (4) below:

[0023] (3)

[0024] (4)

[0025] In formulas (3) and (4), E represents the average value calculated from the data. This represents the model predicting the log probability that a given input sample X belongs to class c. Therefore, during model training, the generator maximizes... - The discriminator is minimized. + A game-theoretic state is reached. The generator produces images by distributing random points in a latent space. The latent space is essentially composed of noise derived from a Gaussian distribution. The model further takes class labels as parameters. In the generator model, several transposed convolutional layers, also known as up-convolutional layers, are used. These layers are interspersed with batch normalization layers. These layers work together to transform the input noise vector into a 3-channel image with 224 × 224 dimensions. Similarly, the discriminator structure consists of conventional convolutional layers, the purpose of which is to reduce the height and width of the feature maps. Then there are two final layers: an adversarial layer that distinguishes between real and fake samples and an auxiliary layer that classifies the image into different classes (based on the initial number of classes). This layout makes it easier and more efficient for the model to extract features from real malware images. Transposed convolutions are used to upsample the feature maps, increasing the feature map size to restore the image resolution. After each convolutional layer, a linear unit activation function corrected by LeakyReLU is used.

[0026] The discriminator model is layered on top of the generator model, and the discriminator layers are initially set to be non-trainable; the discriminator only updates the generator. After completing the malware image generation task, a resampling operation is performed to balance the dataset.

[0027] In this embodiment, the improved GAN architecture differs from traditional GANs in that traditional GANs only use random noise as input and the discriminator performs a single task to judge the authenticity of data, while this improves upon it by introducing class labels as conditional input and adding a class classification branch to the discriminator to form a dual-task structure. Its advantages lie in its ability to accurately generate realistic images of a specified class, and the improved training stability and generation quality through multi-task learning, significantly outperforming the unconstrained generation mode of traditional GANs in terms of controllability and specificity. The use of transposed convolution allows for the learning and preservation of feature information through learnable convolution kernel parameters during the magnification process, enabling the generated fake images to possess both the target size and clear details, exhibiting a style consistent with real data. Furthermore, a hybrid loss function is used to reduce intra-class distance and increase inter-class distance, thereby generating higher-quality images.

[0028] S4: Class Distance Optimization: By optimizing the loss function, the class distance is reduced. At the same time, the advantages of feature vector normalization and margin adjustment are combined to further compress the intra-class distance and increase the inter-class distance, thereby improving the stability and discrimination ability of deep learning features. In deep learning, the SoftMax loss function is generally used to deal with multi-class problems. Its expression is shown in the following (5):

[0029] (5)

[0030] Where n represents the total number of samples, and C represents the number of categories. Indicator functions representing the actual labels. This represents the probability that sample i belongs to class j. The predicted probability is calculated for each sample i. and true probability The logarithmic product of the loss functions is summed to obtain the sample loss, and finally the summation and averaging of the losses of all samples yields the final loss. While the SoftMax loss function can effectively compensate for classification, it ignores inter-class and intra-class relationships. Therefore, a hybrid loss function is used to reduce the distance between intra-class samples and increase the distance between inter-class samples, thereby increasing the discriminative power of deep features. middle Indicates sample The score on the true category c, This represents the model's highest score in all categories other than the true category. This is achieved by introducing... The adaptive interval parameter controls the minimum distance between classes. The core of this expression is that when... The model is penalized, forced to score the true class. At least higher than the highest negative class score Otherwise, a loss will occur. Furthermore, a central loss is introduced. Reduce intra-class distance, where This represents a vector of a sample in the feature space. The feature center of category y is represented by y, and The weighting coefficients are used to balance the two losses. Their main function is to constrain the feature x of the same sample to be as close as possible to its class center. This reduces intra-class distance. It involves testing the average distance between samples from the same class and different classes. The distance is used to demonstrate the reliability of the scheme. and The value of is determined by the following method:

[0031] (6)

[0032] (7)

[0033] (8)

[0034] in, It is the average distance between classes. This represents the average distance within each class. C represents the number of classes. This represents the feature center of the i-th category. Let x represent the Euclidean distance between the two classes. This represents the true label corresponding to x. N represents the total number of samples. Finally... depending on and The ratio between them reflects the relative scale of the current model's classification ability. If the inter-class distance is large and the intra-class distance is small, then... It increases, and vice versa. Among them... To prevent division by zero, a very small constant, based on experience... The value is 0.2.

[0035] (9)

[0036] and The value is determined using the warm-up approach. The value is 0.5, where t represents the current epoch number. Used for control The growth rate. This formula represents the initial growth rate at the beginning of training. The loss function is very small at this point, and its primary function is determined by the previous steps. As training gradually stabilizes... Growth to We begin to strengthen intra-class constraints and enhance feature compactness in order to reduce inter-class distance.

[0037] The loss function is shown below:

[0038] (10)

[0039] Therefore, the final loss function is:

[0040] (11)

[0041] The diagrams illustrating the inter-class and intra-class distances are as follows: Figure 2 As shown.

[0042] S5: An expanded dataset is obtained through a fine-tuned GAN framework. This dataset ensures an equal number of samples for each category, avoiding majority bias during model training. The final dataset is then input into the VisionTransformer (ViT), which learns global image features that adapt well to different image resolutions and aspect ratios without requiring extensive architectural adjustments. Furthermore, ViT's self-attention mechanism provides enhanced interpretability; analyzing the attention weights offers insights into the model's predictions. Compared to CNNs, ViT typically exhibits weaker inductive bias, leading to greater reliance on model regularization or data augmentation when training on smaller datasets. We addressed this deficiency earlier by adjusting the GAN model framework and loss function. Additionally, ViT can capture long-range dependencies between different regions in an image, a significant advantage for complex image tasks.

[0043] S6: The ViT model is used to extract features from image blocks, and a multi-head self-attention mechanism is used to extract global features from the feature maps. A CNN is used to extract local features from malware samples. Two-dimensional local information is processed to adapt to the block-based global features, and a NonLocal attention mechanism is used to aggregate the block-based global features to obtain two-dimensional features. The two-dimensional features and global features are then fused to obtain the fused features. The specific fusion process involves two input vectors: one containing extracted local information and the other a position-encoded sequence vector. The input vectors on both sides have different sizes. Let l represent the input of local information, and g represent the position-encoded vector. , Because the global feature vector has an additional `cls_token` compared to the local feature vector, which aggregates information from vectors at other locations, the feature information fusion process skips this token and performs staggered addition with subsequent feature vectors. Then, layer normalization (LN) is applied to the vectors, ensuring the output of each layer remains stable. To accommodate the combination of local and global features, the extracted local features need to be stretched into sequence vectors. However, this may result in the loss of some spatial structure information. By employing the concept of non-local features, these lost spatial relationships can be recovered and strengthened, ensuring that global context awareness is maintained even after transforming local information into one-dimensional sequence vectors, thus avoiding information loss.

[0044] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A malware detection method based on class interval optimization, characterized in that, The method comprises the following steps: Converting the bytecode of the malware to be detected into an RGB three-channel image; Mapping the RGB three-channel image to obtain a high-quality synthetic sample; Inputting the high-quality synthetic sample into a pre-trained malware detection model to obtain a detection result; The pre-training of the malware detection model comprises: obtaining malware sample data, preprocessing the malware sample data; decompiling the preprocessed data to obtain source code files; selecting bytecode features from the source code files and converting the bytecode features into RGB three-channel images; using an optimized GAN architecture to expand the RGB three-channel images to obtain expanded malware samples; inputting the expanded malware samples into a deep neural network to obtain malware family classification results; calculating the loss function of the model according to the malware family classification results, adjusting the parameters of the model, and when the loss function converges, completing the training of the model.

2. The malware detection method based on class interval optimization according to claim 1, characterized in that, Converting the bytecode into an RGB three-channel image comprises: obtaining the byte data of the file according to the bytecode, wherein each byte represents an operation unit; converting the byte data into a matrix and normalizing the matrix; mapping every 3 elements in the normalized matrix to three channels of an image to obtain the corresponding RGB image.

3. The malware detection method based on class interval optimization according to claim 1, characterized in that, Mapping the RGB three-channel image comprises: using a convolutional encoder to encode the input image into a low-dimensional latent vector; wherein a LeakyReLU activation function is introduced in the convolutional encoder, and a ResNet residual block is combined to enhance the feature expression capability.

4. The malware detection method based on class interval optimization according to claim 1, characterized in that, The preprocessing of the malware sample data comprises: judging whether the malware file is armored, and if armored, performing shell searching and decompilation on the malware file.

5. The malware detection method based on class interval optimization according to claim 1, characterized in that, Selecting bytecode features from the source code files and converting the bytecode features into images comprises: reading the byte stream from the decompiled malware sample; obtaining the byte data of the file according to the byte stream, wherein each byte represents an operation unit; converting the byte data into a matrix and normalizing the matrix; mapping every 3 elements in the normalized matrix to three channels of an image to obtain the corresponding RGB image.

6. The malware detection method based on class interval optimization according to claim 1, characterized in that, Expanding the RGB three-channel image using the optimized GAN architecture comprises: obtaining a random noise vector; inputting the random noise vector and the RGB three-channel image into the generator of the optimized GAN model, generating a fake sample with a size of 224x224x3 through multiple rounds of transpose convolution and BatchNorm upsampling; inputting the real sample or the fake sample into the discriminator, completing the adversarial task and the auxiliary classification task through multiple rounds of convolution and LeakyReLU operation; using transpose convolution in the discriminator to realize learnable upsampling, and gradually generating high-resolution images from low-dimensional features.

7. The malware detection method based on class interval optimization according to claim 6, characterized in that, The GAN architecture processes the RGB three-channel image, including: inputting a random noise vector and the RGB three-channel image into the generator for fusion, performing transposed convolution and batch normalization operations on the fused image, and performing upsampling to generate false data consistent with the size of the real data; the discriminator simultaneously receives real data and corresponding class labels and false data and corresponding class labels output by the generator, extracts features by convolution and LeakyReLU operation and flattening, and then outputs "data true or false judgment" and "class judgment" through double branches; the generator and the discriminator alternately play, the generator calculates the generator loss, the discriminator calculates the discriminator loss, and the parameters are updated through loss back propagation, and the iteration is looped until the generator generates a realistic and class-controllable image or the discriminator cannot effectively distinguish true and false data.

8. The malware detection method based on class interval optimization according to claim 1, characterized in that, The deep neural network processes the expanded malware sample, including: using a ViT model to extract features from the image blocks, and using a multi-head self-attention mechanism to extract global features from the feature maps; using a CNN to extract local features from the malware sample; processing two-dimensional local information to adapt to the block global features, using a NonLocal attention mechanism to aggregate the block global features, and obtaining two-dimensional features; fusing the two-dimensional features and the global features to obtain fused features.

9. The malware detection method based on class interval optimization according to claim 1, characterized in that, The loss function is optimized, and the loss function adopts a compound loss function optimized by class interval.

Images