Zero-day malicious code detection model and method and model construction device and method

Abstract

The invention discloses a zero-day malicious code detection model and method and a model construction device and method, and belongs to the technical field of malicious code detection. According to the model construction method, TransXNet serves as a basic model, training optimization is conducted on the basic model through a training set, then the zero-day malicious code detection model is obtained, diffusion operation is conducted on a visual malicious code image through a diffusion model DDIM, a new malicious code image is generated, and therefore the zero-day malicious code detection model is obtained. And combining the new malicious code image with the original malicious code image to jointly form training set data. By the adoption of the construction method, the detection accuracy and robustness of the model for the zero-day malicious code are improved, the problem of data set imbalance is solved, and therefore an effective technical scheme is provided for solving the problems existing in existing zero-day malicious code detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of malicious code detection, and more specifically, relates to a zero-day malicious code detection model, method, and model construction device and method. Background Art

[0002] In today's information age, computers and the Internet have become the key pillars of society. With the acceleration of the digitalization process, cybersecurity threats are becoming increasingly severe. Malicious code, as the main tool for cyber attacks, poses a great threat to the security of individuals, enterprises and even countries. Malicious code is a type of software or script designed to damage the system, steal information or perform other malicious operations. It comes in various forms, including viruses, worms, Trojan horses, etc.

[0003] Among them, zero-day malicious code has become a major challenge in the field of network security because of its characteristics of exploiting undisclosed or unpatched vulnerabilities to attack, with extremely high concealment and destructive power. Victims are usually defenseless before the vulnerability is exposed or fixed, and are extremely vulnerable to attacks. With the continuous evolution of malicious code and network attack methods, traditional security protection measures are increasingly unable to cope with this severe situation. How to develop more advanced malicious code detection technology to improve the accuracy and real-time performance of detection has become a research topic that needs to be urgently solved in the current network security field.

[0004] Due to the unknown and hidden nature of zero-day malicious code, traditional detection methods face huge challenges. Signature-based detection methods rely on known features and are powerless against unknown zero-day malicious code; while behavioral analysis and anomaly detection methods can identify some abnormal behaviors, they are still powerless against advanced zero-day attacks. In recent years, deep learning technology has shown great potential in zero-day malicious code detection with its ability to automatically extract complex features, providing a more efficient detection method.

[0005] Current malicious code detection technologies mainly include the following categories:

[0006] (1) Signature-based detection method: Rapid detection is achieved by matching the signatures of known malicious code. This type of method is widely used in malicious code databases. Its advantages are fast detection speed and low resource consumption. However, it has obvious defects. It mainly relies on known feature libraries and cannot effectively identify zero-day malicious code or new variants of malware. It is difficult to cope with the challenges brought by the continuous evolution of attack methods. Since zero-day malicious code usually exploits unknown vulnerabilities, the victim system is defenseless before the vulnerability is fixed, which seriously limits the signature-based detection method in practical applications.

[0007] (2) Byte sequence-based detection methods: Deep learning models, such as convolutional neural networks (CNNs) or long short-term memory networks (LSTMs), are used to analyze the binary byte stream of malicious code. This type of method improves the generalization ability of detecting malicious code to a certain extent by automatically extracting patterns and features from the byte stream. However, due to the varying lengths of malicious code byte streams and sparse features, the accuracy of detection is insufficient, especially when dealing with complex malware types, the generalization ability of the model is often weak. This is because binary data lacks structured features in form, making it difficult for the model to capture global patterns.

[0008] (3) Detection methods based on behavioral analysis: By monitoring the dynamic behavior of malicious code in a sandbox environment, the running characteristics are analyzed. This type of method has high detection accuracy in specific scenarios and can provide a comprehensive detection perspective through behavioral data. However, its main problem is that it relies on a virtual execution environment, which consumes a lot of resources. At the same time, some advanced malicious codes have the ability to circumvent sandbox technology, making this type of method easy to bypass. In addition, for malicious behaviors that are not executed, behavioral analysis methods are difficult to provide effective detection.

[0009] (4) Detection method based on generative adversarial network (GAN): By generating realistic malware samples to simulate zero-day malicious code, the detection system's ability to identify unknown threats is improved. Although GAN performs well in generating malicious samples and enhancing data, its training process is often unstable, prone to mode collapse problems, and extremely sensitive to hyperparameters, lacking a clear probabilistic interpretation. In addition, the adversarial training between the generator and the discriminator also increases the complexity of the model and the difficulty of application.

[0010] In summary, although the above existing detection methods have made significant progress in malicious code detection, they still expose significant limitations when dealing with the unknown and complexity of zero-day malicious code. The main problems include: insufficient detection capabilities for unknown threats, weak model generalization performance, unstable training process, and dependence on the execution environment. These problems mainly stem from the limitations of existing methods in feature extraction, data sparsity processing, and dynamic behavior monitoring. They are urgently needed to be improved through technological innovation to improve the accuracy, robustness, and real-time performance of malicious code detection. Summary of the invention

[0011] The purpose of the present invention is to provide a zero-day malicious code detection model, method and model construction device and method to solve the problem that the accuracy, robustness and real-time performance of zero-day malicious code detection cannot be guaranteed due to the limitations of existing detection methods in feature extraction, data sparsity processing or dynamic behavior monitoring.

[0012] In order to achieve the above object, the technical solution provided by the present invention is:

[0013] The first aspect of the present invention provides a method for constructing a zero-day malicious code detection model, taking TransXNet as a basic model, training and optimizing it to obtain the zero-day malicious code detection model, and the method specifically includes:

[0014] Preprocessing the malicious code sample data to obtain training set data; and

[0015] The TransXNet is trained and optimized using the training set data to obtain the detection model;

[0016] The preprocessing of the malicious code sample data includes: using a diffusion model DDIM to perform a diffusion operation on the visualized malicious code image to generate a new malicious code image, and the new malicious code image and the original malicious code image together constitute the training set data.

[0017] In order to address the shortcomings of traditional methods in zero-day malicious code detection, the present invention uses TransXNet as the basic model and utilizes the diffusion model DDIM to generate high-quality malicious code images. The generated new malicious code images are combined with the original malicious code images to form a training set data for training TransXNet, which not only improves the model's detection accuracy and robustness for zero-day malicious code, but also improves the problem of data set imbalance, thereby providing an effective technical solution to solve the problems existing in existing zero-day malicious code detection.

[0018] According to the construction method of any technical solution described in the first aspect of the present invention, the diffusion model DDIM is used to perform a diffusion operation on the visualized malicious code image to generate a new malicious code image, which specifically includes: adding noise to the data sample to gradually turn it into pure noise, and then gradually restoring the details and structure of the image through the inverse denoising process obtained through training, thereby generating a new complex zero-day malicious code image.

[0019] According to the construction method of any technical solution described in the first aspect of the present invention, the preprocessing of malicious code sample data further includes:

[0020] Performing visualization processing on the malicious code to obtain a visualization image of the malicious code;

[0021] And / or further perform image size normalization processing on the malicious code visualization image.

[0022] According to the construction method of any technical solution described in the first aspect of the present invention, the image size normalization process is performed using a bicubic interpolation algorithm and a sharpening technique, and / or the malicious code is visualized, including:

[0023] Convert the binary file of malicious code into grayscale image;

[0024] Convert assembly code files to RGB images;

[0025] By fusing the grayscale image with the RGB image, we can get a visualization image of the malicious code.

[0026] To ensure the integrity and feature diversity of malicious code information, the present invention adopts an improved malicious code visualization method, that is, the grayscale image generated by the .bytes file is merged with the RGB image generated by the opcode feature map extracted from the .asm file. By integrating information from different sources, the diversity of features is significantly improved, and the comprehensiveness of feature extraction is enhanced, providing a solid foundation for achieving efficient and accurate detection.

[0027] By combining the bicubic interpolation algorithm with the sharpening technology, the malicious code visualization image is further normalized in size, which not only ensures that the texture features remain unchanged during the normalization process, but also improves the image clarity, making the normalized image more suitable for subsequent malicious code detection tasks. It can effectively solve the problem of texture feature changes that may be caused by the normalization process of the malicious code image.

[0028] According to the construction method of any technical solution described in the first aspect of the present invention, the calculation formula of the loss function Focal Loss of the TransXNet model is:

[0029] FL(p t )=-α t (1-p t ) γ log(p t )

[0030] Among them, p t is the model’s predicted probability for the true category, α t is a balancing parameter used to balance the ratio of positive and negative samples, and γ is an adjustment parameter used to adjust the weight of difficult and easy samples.

[0031] The present invention further improves the cross entropy loss function of the original model and adopts Focal Loss as the loss function. Focal Loss significantly improves the classification performance of the model under data imbalance conditions by reducing the influence of easy-to-classify samples and increasing the weight of difficult-to-classify samples.

[0032] According to the construction method of any technical solution described in the first aspect of the present invention, the TransXNet adopts an improved TransXNet model, which includes:

[0033] Multi-scale Attention Module (EMA), which is used to extract features from the input sequence through multi-scale convolution and pyramid pooling, and perform feature fusion through the cross-spatial attention mechanism;

[0034] Dynamic position encoding module DPE, used to dynamically generate position information of input sequence;

[0035] A lightweight dual dynamic token mixer D-Mixer, which is used to extract global and local dynamic information from the fused features; and

[0036] Multi-scale feedforward network MS-FFN is used to introduce multi-scale features so that the model can combine information of different scales during feature extraction.

[0037] EMA divides the input image into blocks and embeds it into a high-dimensional space, providing a basis for subsequent feature extraction, and outputs the embedded features for subsequent dynamic position encoding (DPE) and feature mixing module (D-Mixer) processing. DPE injects position information into the input features to enhance the spatial relationship modeling capability. It is located at the beginning of each stage, and the processed features are passed to the feature mixing module (D-Mixer). D-Mixer continuously integrates contextual information during feature processing and passes the results to the multi-scale feedforward network (MS-FFN). MS-FFN receives features from D-Mixer, performs further cross-channel and multi-scale feature aggregation, and then passes the results to the output layer.

[0038] As a preferred implementation, in order to reduce the model computational overhead and improve detection efficiency, the present invention introduces a multi-scale attention module EMA based on the existing TransXNet model. EMA improves the performance of the model by enhancing multi-scale feature integration, context perception and noise suppression capabilities; at the same time, the EMA module can effectively capture features of different scales, enhance the recognition ability of diverse features, and accurately detect malicious code in complex scenarios while reducing computational overhead.

[0039] According to the construction method of any technical solution described in the first aspect of the present invention, the multi-scale attention module EMA includes two parallel sub-networks, which extract and fuse features of the input sequence, specifically including:

[0040] Divide the input feature map X into multiple sub-feature groups;

[0041] The two sub-networks respectively process the sub-feature groups using different convolution kernels to obtain output feature maps of the two sub-networks;

[0042] The output feature maps of the two sub-networks are then fused through the cross-spatial attention mechanism.

[0043] A second aspect of the present invention provides a zero-day malicious code detection model construction device, comprising:

[0044] A training set data preprocessing unit, the unit being used to preprocess the malicious code sample data to obtain training set data; and

[0045] A model training optimization unit, which is used to train and optimize TransXNet using the preprocessed training set data to obtain the detection model;

[0046] The training set data preprocessing unit includes a diffusion module, which is used to perform a diffusion operation on the visualized malicious code image using a diffusion model DDIM to generate a new malicious code image, and the new malicious code image and the original malicious code image together constitute the training set data.

[0047] According to the device of any technical solution described in the second aspect of the present invention, the diffusion model DDIM is used to perform a diffusion operation on the visualized malicious code image to generate a new malicious code image, specifically including: adding noise to the data sample to gradually turn it into pure noise, and then gradually restoring the details and structure of the image through the inverse denoising process obtained through training, thereby generating a new complex zero-day malicious code image.

[0048] The third aspect of the present invention provides a zero-day malicious code detection model, which is constructed using the method of any technical solution described in the first aspect of the present invention.

[0049] A fourth aspect of the present invention provides another zero-day malicious code detection model, which adopts an improved TransXNet model and includes:

[0050] Multi-scale Attention Module (EMA), which is used to extract features from the input sequence through multi-scale convolution and pyramid pooling, and perform feature fusion through the cross-spatial attention mechanism;

[0051] Dynamic position encoding module DPE, used to dynamically generate position information of input sequence;

[0052] A lightweight dual dynamic token mixer D-Mixer, which is used to extract global and local dynamic information from the fused features; and

[0053] Multi-scale feedforward network MS-FFN is used to introduce multi-scale features so that the model can combine information of different scales during feature extraction.

[0054] By improving the existing TransXNet model and introducing the multi-scale attention module EMA, we can effectively capture features of different scales, enhance the ability to recognize diverse features, and accurately detect malicious code in complex scenarios while reducing computational overhead.

[0055] According to the detection model of any technical solution described in the fourth aspect of the present invention, the multi-scale attention module EMA includes two parallel sub-networks, which extract and fuse features of the input sequence, specifically including:

[0056] Divide the input feature map X into multiple sub-feature groups;

[0057] The two sub-networks respectively process the sub-feature groups using different convolution kernels to obtain output feature maps of the two sub-networks;

[0058] The output feature maps of the two sub-networks are then fused through the cross-spatial attention mechanism.

[0059] A fifth aspect of the present invention provides a zero-day malicious code detection method, comprising:

[0060] Preprocess the zero-day malicious code to be detected to obtain a visualized malicious code image;

[0061] The visualized malicious code image is input into any zero-day malicious code detection model described in the third aspect or the fourth aspect of the present invention, or constructed using the method described in the first aspect, to classify and detect the malicious code image and obtain a zero-day malicious code detection result.

[0062] According to the zero-day malicious code detection method of any technical solution described in the fifth aspect of the present invention, the pre-processing of the zero-day malicious code to be detected includes malicious code visualization processing and / or image size normalization processing, wherein:

[0063] The visualization processing specifically includes:

[0064] Convert the binary file of malicious code into grayscale image;

[0065] Convert assembly code files to RGB images;

[0066] The grayscale image is merged with the RGB image to obtain a visualization image of the malicious code;

[0067] The image size normalization process is performed using a bicubic interpolation algorithm and a sharpening technique.

[0068] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0069] (1) The present invention uses TransXNet as the basic model. When training and optimizing the model, the new malicious code image samples are generated by performing diffusion operations on the visualized sample images, and are combined with the original samples for model training. This method effectively simulates unknown zero-day malicious code, significantly improves the accuracy and robustness of the detection model in zero-day malicious code detection, and makes up for the shortcomings of the existing technology in processing unknown attack samples.

[0070] (2) The present invention significantly improves the information integrity and feature diversity in the malicious code visualization process by fusing the grayscale image generated by the .bytes file with the RGB image generated by the opcode mapping of the .asm file. Compared with the traditional method, the present invention excels in the comprehensiveness of malicious code feature extraction, can capture richer feature information, and improves the precision and accuracy of detection.

[0071] (3) The present invention combines the bicubic interpolation algorithm with the sharpening technology to perform size normalization processing on the malicious code visualization image, thereby ensuring the retention of texture features during the normalization process and improving the image clarity. This improvement enables the malicious code detection model to more accurately identify malicious code features and ensure the quality of the image after normalization, effectively solving the problem of texture feature changes in the malicious code image during the normalization process in the prior art.

[0072] (4) The present invention further improves the TransXNet model by introducing a multi-scale attention module (EMA). By enhancing the integration of multi-scale features and context perception, the model's ability to identify malicious code in complex scenarios is effectively improved. Compared with the prior art, the present invention not only performs better in diverse feature recognition, but also reduces computational overhead and significantly improves detection efficiency.

[0073] (5) The present invention further adopts Focal Loss as an improved loss function. Focal Loss can effectively reduce the loss of easy-to-classify samples and focus on difficult-to-classify samples, thereby improving the performance of the model on unbalanced data sets and enhancing the classification accuracy in malicious code detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A visualization flow chart of malicious code according to an embodiment of the present invention;

[0075] Figure 2 A comparison chart of different normalization processes for malicious code visualization images;

[0076] Figure 3 This is a schematic diagram of the EMA module structure of an embodiment of the present invention;

[0077] Figure 4 This is a structural diagram of the improved TransXNet in an embodiment of the present invention;

[0078] Figure 5 A schematic diagram of the construction process of TransDiffusionNet in an embodiment of the present invention;

[0079] Figure 6 Schematic diagram of Cross Entropy Loss confusion matrix of an embodiment of the present invention;

[0080] Figure 7 Schematic diagram of Focal Loss confusion matrix of an embodiment of the present invention;

[0081] Figure 8 Schematic diagram of a confusion matrix for a zero-day malicious code detection experiment according to an embodiment of the present invention. DETAILED DESCRIPTION

[0082] In order to further understand the content of the present invention, the present invention is now described in detail in conjunction with specific embodiments.

[0083] Combination Figure 5 As shown, an embodiment of the present invention provides a method for constructing a zero-day malicious code detection model, taking TransXNet as a basic model, training and optimizing it to obtain the zero-day malicious code detection model, and the method specifically includes:

[0084] Preprocessing the malicious code sample data to obtain training set data; and

[0085] The TransXNet is trained and optimized using the training set data to obtain the detection model;

[0086] The preprocessing of the malicious code sample data includes: using a diffusion model DDIM to perform a diffusion operation on the visualized malicious code image to generate a new malicious code image, and the new malicious code image and the original malicious code image together constitute the training set data.

[0087] The diffusion model can generate high-quality and diverse samples by defining the forward noise addition process and the reverse denoising generation process. Compared with traditional methods, the diffusion model has significant advantages in terms of generation stability, sample diversity, and robustness of the training process.

[0088] The core idea of ​​the diffusion model is to gradually add noise to the data samples until the samples become pure noise, and then use the learned reverse process to gradually denoise and restore the original data. The whole process is divided into two stages: forward diffusion and reverse generation. In the forward diffusion stage, the structure and characteristics of the data samples are gradually covered by noise and eventually transformed into pure noise; in the reverse generation stage, the model gradually restores the original characteristics of the data through denoising and generates high-quality samples from the noise. This two-stage mechanism ensures the high quality and diversity of the generated samples, while greatly improving the stability and reliability of model training. The specific process is as follows:

[0089] The forward diffusion process is a process of gradually adding noise. Given a data sample a0, the forward process generates a series of samples a1, a2, a3, ..., a t , where a small amount of Gaussian noise is added at each step.

[0090]

[0091] Among them, β t is the predefined noise scheduling parameter, represents the normal distribution, Ι represents the unit matrix, q(a t |a t-1 ) represents the forward process transition probability.

[0092] The reverse generation process is a step-by-step denoising process, aiming to generate t Restore to the original data b0. This process is done through a parameterized model p θ (b t-1 |b t ) to achieve it.

[0093] p θ (b t-1 ∣b t )=N(b t-1 ;μ θ (b t ,t),Σ θ (b t ,t))

[0094] Among them, θ is the parameter of the model, μ θ (b t ,t) and ∑ θ (b t ,t) are the mean and covariance of the neural network predictions, represents a normal distribution. θ (b t-1 |t) is the sample b at the given t-th step t Under the condition of t-1The conditional probability distribution of .

[0095] DDIM is an improved diffusion model that accelerates the data generation process by introducing a new deterministic sampling process. Compared with the traditional diffusion model, DDIM significantly reduces the generation time while maintaining the generation quality. The core formula of the DDIM reverse process is as follows:

[0096]

[0097] Among them, x t is the variable at step t, α t and α t-1 are the scaling factors at step t and t-1 respectively. ∈ θ (x t ,t) is the noise predicted by the neural network, σ t is a noise-related parameter that controls the noise level at each step.

[0098] DDIM generates images by step-by-step inference. Starting from the initial time step, the uncertainty of random variables is gradually reduced by continuously iterating the dynamic system model, so as to generate high-quality images more efficiently. The present invention generates new malicious code images by applying the DDIM diffusion operation to the visualized image. This process effectively simulates unknown malicious code samples and combines the generated samples with the original samples for training the malicious code detection model TransXNet. By introducing simulated malicious code samples, the model performs better in detecting and classifying zero-day malicious code, significantly improving accuracy and robustness. In addition, the generated new samples help to balance the data set and reduce the impact of uneven majority and minority class samples, providing an effective solution for zero-day malicious code detection.

[0099] In some embodiments, the preprocessing of the malicious code sample data further includes: visualizing the malicious code to obtain a visualized image of the malicious code; combining Figure 1 , the visualization process preferably comprises the following steps:

[0100] Convert the binary file (.bytes file) of the malicious code into a grayscale image;

[0101] Convert assembly code files to RGB images;

[0102] By fusing the grayscale image with the RGB image, we can get a visualization image of the malicious code.

[0103] Among them, the binary file (.bytes file) of the malicious code is converted into a grayscale image, and its specific method is further preferably: each 8-bit unsigned integer is read in groups and converted into a decimal integer. Since the malicious code files are of different sizes, the traditional method of determining the image size by the file size will cause information loss. For this reason, the application calculates the length and width of the image according to the number of file bytes to ensure that the size is sufficient to accommodate all the data. Subsequently, the data is converted into a two-dimensional array, each element corresponds to a grayscale value, and a grayscale image is generated.

[0104] The method of converting an assembly code file (.asm file) into an RGB image is as follows: parsing the assembly file line by line, extracting the opcodes and counting their frequencies, generating an opcode frequency feature vector with a length of 256; normalizing the frequencies to 0 to 255, and if the vector length is not a multiple of 3, padding with 0 to form a three-channel array; finally, constructing an RGB array and generating an RGB image.

[0105] The fusion of the grayscale image and the RGB image is specifically as follows: first, the grayscale image is converted to RGBA format, and the grayscale value is used as the transparency information of the Alpha channel; then, the RGB image is adjusted to the same size as the grayscale image to ensure correct alignment; the RGB image is synthesized with the grayscale image with transparency information to generate a fused image. This fusion combines the structural information of the grayscale image with the semantic information of the RGB image, thereby providing richer features.

[0106] The fusion of grayscale and RGB images not only integrates information from different sources, but also improves the recognition ability of the model, especially when dealing with obfuscated, variant, and packed malicious code, it can better capture key features and reduce the probability of false positives and false negatives. In addition, this method enriches the diversity of samples, reduces overfitting, and improves the generalization ability of the model on unseen data.

[0107] In some embodiments, the preprocessing of the malicious code sample data further includes: further normalizing the size of the malicious code visualization image. The image size normalization is further performed by using a bicubic interpolation algorithm and a sharpening technique.

[0108] Since the number of bytes of each malicious code file is different, the size of the visualized image is also different. Deep learning models usually require the input image to have a fixed size, so the difference in image size may lead to inconsistent feature map sizes, increase the complexity of model design and training, and affect memory allocation and computing efficiency, and may even hinder training. Therefore, it is necessary to normalize the visualized malicious code image.

[0109] Malicious code images contain a large number of important texture features, which are crucial for the model to effectively extract and identify key information in malicious code. Although malicious code often evades traditional detection methods through variants or modifications, its basic texture features are usually consistent. Therefore, accurately extracting and analyzing these texture features is crucial to dealing with malicious code variants. In order to keep the original texture features of the normalized malicious code image as much as possible, traditional methods usually use bilinear interpolation to adjust the image size. However, experiments show that even if bilinear interpolation is used, some texture features may still be lost or changed. To this end, an embodiment of the present invention proposes a method combining a bicubic interpolation algorithm and a sharpening technique. The bicubic interpolation algorithm can effectively retain details and edge information when scaling, and the sharpening technique further enhances the clarity of the image. This method not only effectively solves the problem of texture feature changes during normalization, but also makes the normalized image clearer and highlights key textures, thereby improving the accuracy and robustness of the model in detecting and classifying malicious code.

[0110] like Figure 2 The figure shows the comparison of different normalization processes of the malicious code visualization image, where (A) is the original image, (B) is the image normalized using the bilinear interpolation algorithm, and (C) is the image generated using the normalization method of the embodiment of the present invention. Figure 2 It can be clearly seen that the texture features of the malicious code image in (B) have changed significantly, which may cause the detection model to learn inaccurate features and affect the accuracy of malicious code detection. However, (C) maintains the texture features of the original image, providing the model with more reliable feature input, thereby improving the detection accuracy and robustness of the model.

[0111] In some embodiments, the TransXNet model directly adopts the existing TransXNet model. TransXNet combines the local feature extraction advantages of CNN and the global feature modeling capabilities of Transformer. It can efficiently extract features and deeply understand semantics in malicious code detection, thereby significantly improving detection performance and applicability.

[0112] Due to its static characteristics, traditional convolution is difficult to dynamically adapt to input changes, resulting in a difference in representation capabilities between convolution and self-attention. To this end, TransXNet introduces a lightweight dual dynamic token mixer (D-Mixer), which combines the global self-attention mechanism (Overlapping Spatial Reduction Attention, OSRA) and input-dependent deep convolution (Input-dependent Depthwise Convolution, IDConv). While maintaining input dependency, D-Mixer uses a wide receptive field and inductive bias to powerfully capture global and local dynamic information. Specifically, D-Mixer divides the input feature map into two parts: one part is processed by the global self-attention module, and the other part is processed by the input-dependent deep convolution. This dual processing mechanism greatly reduces the computational overhead while enhancing the generalization ability of the model and ensures that the performance is effectively improved.

[0113] The Dynamic Position Embedding (DPE) module is an important mechanism in TransXNet for dynamically generating position information of input sequences. Traditional Transformer models use predefined sine and cosine functions as static position encodings, which cannot be adjusted according to the specific conditions of the input. DPE can dynamically generate position encodings while the model is running, better capturing the complex relationships and structural changes in the input sequence.

[0114] The Multi-scale Feed-forward Network (MS-FFN) introduces a multi-scale feature processing mechanism, which enables the model to combine information of different scales during feature extraction. Compared with traditional feed-forward networks, MS-FFN can more comprehensively characterize the complexity of input data, making the model more robust when dealing with interference such as deformation and noise, thereby improving the robustness and expressiveness of the model.

[0115] Overall, TransXNet, through its innovative module design, not only enhances the model's feature extraction capabilities, but also significantly optimizes the model's computational efficiency and accuracy, enabling it to perform well in malicious code detection tasks.

[0116] In order to effectively integrate features of different scales, improve the model's ability to capture various features, and enhance the perception of contextual information so that the model can still accurately identify malicious code in complex backgrounds, the TransXNet model is optimized in some embodiments, that is, a multi-scale attention module EMA is introduced on the basis of the above-mentioned existing TransXNet model, and features of different scales are integrated in combination with a cross-space learning mechanism. The EMA module extracts features through multi-scale convolution and pyramid pooling, and then fuses them through a cross-space attention mechanism, which not only captures details but also understands the global context. In addition, the EMA module has noise suppression capabilities, which improves classification performance in complex backgrounds, especially when dealing with variant, obfuscated and packed malicious code.

[0117] To reduce computational complexity, the EMA module reshapes some channels into batch dimensions and groups the channel dimensions into multiple sub-features. Specifically, Figure 3 As shown, the EMA module groups the input feature map X into G sub-features, each sub-feature X i Contains several channels.

[0118] X=[X1,X2,…,X G ]

[0119] The EMA module consists of two parallel sub-networks, one of which uses a 1×1 convolution kernel to process features, and the other uses a 3×3 convolution kernel to process features. The output feature maps of each sub-network are represented as Y1 and Y2 respectively.

[0120] Y1=Conv 1×1 (X)

[0121] Y2=Conv 3×3 (X)

[0122] In each parallel sub-network, the EMA module captures pixel-level pairwise relationships through cross-dimensional interactions. Specifically, it encodes global information for each sub-feature map, recalibrates channel weights, and fuses the output features of two parallel sub-networks. Through a cross-space learning method, the EMA module achieves local cross-channel interactions without reducing the channel dimension. The multi-scale parallel design of this module effectively enhances the context-awareness of the model, enabling it to better understand the relationship between different regions in the image, thereby improving the accuracy and robustness of image classification tasks.

[0123] In the TransXNet model after applying the EMA module, the input feature map is divided into multiple sub-feature groups, and these sub-features are processed by different convolution kernels to achieve effective integration of multi-scale information. This mechanism significantly enhances the context perception ability of the model, thereby improving the accuracy and robustness of classification. In various image classification tasks, the EMA module not only greatly reduces the computational complexity, but also improves the detection efficiency, especially in complex malicious code identification tasks, showing excellent performance and adaptability. The TransXNet model after applying the EMA module is as follows Figure 4 shown.

[0124] In order to further eliminate the impact of data set imbalance on model performance, in some embodiments, the loss function of the TransXNet model is further optimized, and the calculation process of the loss function Focal Loss is shown in the following formula:

[0125] FL(p t )=-α t (1-p t ) γ log(p t )

[0126] Among them, p t is the model’s predicted probability for the true category, α t is a balancing parameter used to balance the ratio of positive and negative samples, and γ is an adjustment parameter used to adjust the weight of difficult and easy samples.

[0127] In classification tasks, positive and negative samples are often unbalanced. To solve this problem, Focal Loss introduces a balance parameter α t , weight the loss of positive and negative samples with different weights. If the ratio of positive and negative samples is seriously unbalanced, you can set α t The value of γ can be used to increase the focus on the minority class. By adjusting the value of γ, the weight distribution of easy samples and difficult samples can be controlled. When γ = 0, Focal Loss degenerates into ordinary cross entropy loss. As γ increases, the loss weight of easy samples gradually decreases, making the model pay more attention to difficult samples. Through this mechanism, Focal Loss effectively improves the performance of the model on unbalanced data sets, especially in the classification accuracy of minority class samples.

[0128] As a further preferred embodiment of any embodiment of the present invention, the calculation formula of the loss function FocalLoss of the TransXNet model is:

[0129] FL(p t )=-α t (1-p t )γ log(p t )

[0130] Among them, p t is the model’s predicted probability for the true category, α t is a balancing parameter used to balance the ratio of positive and negative samples, and γ is an adjustment parameter used to adjust the weight of difficult and easy samples.

[0131] The embodiment of the present invention also provides a zero-day malicious code detection model construction device, including:

[0132] A training set data preprocessing unit, the unit being used to preprocess the malicious code sample data to obtain training set data; and

[0133] A model training optimization unit, which is used to train and optimize TransXNet using the preprocessed training set data to obtain the detection model;

[0134] The training set data preprocessing unit includes a diffusion module, which is used to perform a diffusion operation on the visualized malicious code image using a diffusion model DDIM to generate a new malicious code image, and the new malicious code image and the original malicious code image together constitute the training set data.

[0135] As a further preferred embodiment, the training set data preprocessing unit also includes a visualization processing module and / or an image size normalization processing module, wherein the specific processing method of the visualization processing module and / or the image size normalization processing module is preferably to adopt any of the visualization and normalization processing methods described above, which will not be introduced in detail here.

[0136] It should be noted that the above-mentioned TransXNet model can directly adopt the existing TransXNet model, or can adopt the improved TransXNet model in the present invention, which will not be introduced in detail here.

[0137] An embodiment of the present invention further provides a zero-day malicious code detection model, which is constructed using the model construction method of any of the above embodiments of the present invention.

[0138] The embodiment of the present invention also provides a zero-day malicious code detection method, including:

[0139] Preprocess the zero-day malicious code to be detected to obtain a visualized malicious code image;

[0140] The visualized malicious code image is input into any of the above embodiments, or a zero-day malicious code detection model is constructed using the construction method in any of the above embodiments to classify and detect the malicious code image to obtain a zero-day malicious code detection result.

[0141] As a preferred implementation method, the preprocessing of the zero-day malicious code to be detected includes malicious code visualization processing and / or image size normalization processing. The malicious code visualization processing and image size normalization processing method are further preferably to adopt the visualization and normalization processing method described in the present invention, which will not be repeated here.

[0142] Model performance test experiment:

[0143] (1) Experimental environment and parameter settings

[0144] In order to evaluate the performance of the improved model (TransDiffusionNet), a zero-day malicious code detection platform based on deep learning was built in the experiment. The experimental environment configuration is shown in Table 1.

[0145] Table 1 Experimental environment configuration

[0146]

[0147]

[0148] The experimental model is built based on the PyTorch framework. The core parameters and parameter values ​​during the model training process are shown in Table 2.

[0149] Table 2 Experimental parameter configuration

[0150]

[0151] (2) Dataset Introduction

[0152] In order to evaluate the performance of the model on different data sets, this application selected the Malimg data set and the Microsoft malicious code data set as training data sets, but other malicious code image data sets that can be visualized or directly visualized can also be used.

[0153] The Malimg dataset is a well-known dataset specifically used for malware classification research. It contains 9,339 images from 25 different malware families. These images are generated by converting the binary code of the malware into grayscale images, and each image represents a malware sample. The notable feature of the Malimg dataset is its visual diversity, with different malware families showing unique textures and patterns in the images. This diversity makes the dataset particularly suitable for testing the performance of models in identifying and distinguishing different types of malware. Due to the imbalance in the number of samples between families, the Malimg dataset also poses additional challenges to the model's ability to handle data skew and sample scarcity. The specific composition, family categories, and sample distribution of the Malimg dataset are shown in Table 3.

[0154] Table 3 Malimg dataset

[0155]

[0156]

[0157] The Microsoft Malicious Code Dataset, also known as the BIG 2015 dataset, is a large malware classification challenge dataset released by Microsoft in 2015. The dataset includes 9 malicious code families and a total of 10,868 samples. The dataset consists of two types of files: one is .byte files, which represent malicious code binary files without PE headers; the other is .asm files, which represent malicious code instruction sets obtained by disassembly. These files provide rich structured information and can be used to generate multi-dimensional malicious code images. The BIG 2015 dataset is characterized by covering a wider range of malware types, and there is also an imbalance in the number of family samples within the dataset. For example, the malicious code family Simda has the least number of samples, only 42, accounting for less than 1% of the total number of samples, which makes this dataset an ideal tool for evaluating the performance of models in complex and diverse scenarios. The specific composition, family categories and sample distribution of the BIG 2015 dataset are shown in Table 4.

[0158] Table 4 Microsoft malicious code dataset

[0159]

[0160] The combined use of these two datasets can comprehensively examine the model's robustness, generalization ability, and ability to handle unbalanced datasets in malicious code classification tasks.

[0161] (3) Evaluation indicators

[0162] This application comprehensively evaluates the performance of the TransDiffusionNet model through four indicators: accuracy, precision, recall, and F1-score. Accuracy measures the proportion of samples predicted correctly by the model, reflecting the overall classification effect of the model. Precision indicates the proportion of samples predicted to be positive that are actually positive, evaluating the accuracy of the model in positive prediction. Recall indicates the proportion of samples that are actually positive that are correctly identified as positive, reflecting the detection ability of the model. The F1 score, as the harmonic mean of precision and recall, comprehensively weighs the two and is suitable for evaluating classification performance under imbalanced data sets.

[0163]

[0164] Among them, TP (True Positive): the number of samples predicted to be positive and actually positive, TN (True Negative): the number of samples predicted to be negative and actually negative, FP (False Positive): the number of samples predicted to be positive but actually negative, FN (False Negative): the number of samples predicted to be negative but actually positive.

[0165] This application uses FID (Fréchet Inception Distance) as an evaluation indicator for the quality of images generated by the diffusion model. It measures the similarity between the generated image and the real image by calculating the statistical distance between them in the feature space. FID evaluates the quality and diversity of generated images by calculating the statistical distribution distance between the real image and the generated image in the feature extraction layer of the pre-trained Inception v3 model (which converts the image into a high-dimensional feature representation to evaluate the distribution difference between the generated image and the real image, thereby measuring the quality of the generated image).

[0166]

[0167] Among them, μ R , μ G represents the mean vector of features extracted from real data and generated data, ∑ R ,∑ G represents the covariance matrix of features extracted from real data and generated data, ‖μ R -μ G ‖ 2It represents the square of the Euclidean distance between two mean vectors, which is used to measure the difference between the generated image and the real image in the feature space. The square root of the product of two covariance matrices is used to measure the similarity between the covariance matrices of the generated image and the real image. This formula measures the statistical distance between the generated image and the real image in the feature space, that is, their distribution difference. The lower the FID value, the better the quality of the generated image and the closer it is to the distribution of the real image.

[0168] (4) Malicious code visualization improvement experiment

[0169] In order to ensure the integrity of information and the diversity of features during the visualization of malicious code, improve the comprehensiveness of feature extraction, and thus enhance the performance of the model in malicious code detection tasks, this application fuses the grayscale image generated by the .bytes file in the BIG 2015 dataset with the RGB image of the opcode feature map extracted from the .asm file. Based on the TransXNet model, the experiment compares the traditional method of determining the image size based on the file size and the method proposed in this application to calculate the image size based on the number of bytes in the file content (the method before improvement in the table) to verify the impact of information integrity on model performance. In addition, the experiment also compares the grayscale image before fusion and the image after fusion (the improved image in the table is the image whose length and width are calculated based on the number of bytes in the file content, and the fused image refers to the image after the grayscale image and the RGB image are fused during visualization) to further verify the effect of feature diversity on improving model performance.

[0170] Table 5 Comparative experiments of visualization methods

[0171]

[0172] The experimental results in Table 5 show that the improved visualization method significantly improves the performance of the TransXNet model. Compared with the traditional method of determining the fixed image size, the strategy of dynamically calculating the image size based on the number of file bytes not only effectively improves the detection performance of the model, but also optimizes the information integrity of the input data. In particular, the fusion of grayscale images with RGB images of opcode feature maps extracted from .asm files further improves the accuracy and generalization ability of malicious code detection. The experimental results show that the integrity of information and the diversity of features play a key role in improving the performance of the model. By comprehensively utilizing multi-source features, the model not only achieves a significant breakthrough in detection accuracy, but also enhances its adaptability to complex and mutated malicious code.

[0173] (5) Experiment on improving normalization method

[0174] In order to solve the problem that texture features may change during image normalization, this application uses the BIG 2015 dataset based on the TransXNet model and adopts the bicubic interpolation algorithm and sharpening technology to normalize the malicious code visualization image. The experiment compares the traditional bilinear interpolation algorithm with the improved normalization method to evaluate the effect of the improvement of the normalization method on the performance of the model.

[0175] Table 6 Normalization method comparison experiment

[0176]

[0177] The traditional bilinear interpolation algorithm has the problem of changing texture features when processing malicious code image normalization, which limits the performance of the model. To solve this problem, this application proposes a new method that combines the bicubic interpolation algorithm and sharpening technology. The experimental results show that the improved method significantly improves the performance of the model, and the accuracy of the model reaches 99.17%, which is a significant improvement compared to the 98.8% of the traditional method. This shows that the improved normalization method is effective in maintaining image clarity and enhancing detection robustness, and effectively solves the negative impact of texture feature changes on model performance.

[0178] (6) Model improvement experiment

[0179] In order to effectively integrate features of different scales, improve the model's ability to capture various features, and enhance the perception of contextual information so that the model can still accurately identify malicious code in complex backgrounds, the experiment uses TransXNet as the basic model and introduces the multi-scale attention module EMA. In order to comprehensively evaluate the impact of the multi-scale attention module EMA on the TransXNet model, the experiment compares the model performance changes before and after the application of EMA.

[0180] Table 7 TransXNet model improvement experiment

[0181]

[0182] The experimental results in Table 7 show that the TransXNet model with the EMA module has significant improvements in all four indicators, all reaching 99.26%, which is an excellent performance compared to 99.17% without the EMA module. This result shows that the introduction of the EMA module not only strengthens the multi-scale feature integration capability of the model, but also significantly improves the accuracy and stability of malicious code detection while reducing noise interference, and improves the adaptability of the model in dealing with diverse and complex malicious code scenarios.

[0183] (7) Loss function improvement experiment

[0184] In order to eliminate the impact of dataset imbalance on model performance, the experiment improved the loss function and used the BIG2015 dataset to evaluate the performance of Focal Loss, MultiMarginLoss, and Cross Entropy Loss on the TransXNet model. The results are shown in Table 8.

[0185] Table 8 Loss function improvement experiment

[0186]

[0187] Focal Loss alleviates the impact of dataset imbalance on model training by giving higher weights to samples that are difficult to classify, effectively improving the model's ability to detect malicious code. Experimental results show that the introduction of Focal Loss enables the model to better handle samples that are difficult to classify, further improving the accuracy and comprehensiveness of detection. This result verifies the effectiveness of Focal Loss in improving model performance, especially when faced with unbalanced datasets.

[0188] By comparison Figure 6 and Figure 7 From the confusion matrix in Figure 2, we can see that after using the Focal Loss loss function, the detection accuracy of Simda, the smallest malicious code family in the BIG2015 dataset, has increased by 13%, indicating that Focal Loss can effectively enhance the model's ability to recognize small sample categories when dealing with category-imbalanced datasets, further verifying its advantage in dealing with data imbalance problems.

[0189] (8) Diffusion model image generation experiment

[0190] The malicious codes of seven families, Gatak, Kelihos_ver1, Obfuscator.ACY, Ramnit, Simda, Tracur, and Vundo, in the BIG 2015 dataset were used as training datasets. The diffusion model DDIM was used as the generation model, and the quality of the generated images when the generative adversarial network DCGAN and the diffusion model DDPM were used as the generation model were compared.

[0191] Table 9 Generative model comparison experiment

[0192]

[0193] From the experimental results in Table 9, it can be seen that the diffusion model DDIM effectively overcomes the training instability and mode collapse problems of the generative adversarial network, especially achieving the lowest FID scores at both 128×128 and 224×224 image sizes, indicating that its generated images have the best quality.

[0194] (9) Zero-day malicious code detection experiment

[0195] The samples generated by DDIM are treated as unknown malicious code samples to simulate zero-day malicious code. These generated samples are combined with the original samples and then trained using the improved TransXNet model to evaluate the model's ability to detect zero-day malicious code. The experimental results are shown in Table 10.

[0196] Table 10 Zero-day malicious code detection experiment

[0197]

[0198] The experimental results show that when the samples generated by DDIM are introduced, the accuracy of the model is improved to 99.54%, showing the advantage of DDIM in generating high-quality samples. Further application of data enhancement technology (used to expand the training set by generating variant data, improve the generalization ability of the model, and prevent overfitting. Especially when the number of samples is limited, data enhancement technology can simulate the diversity of data and make the model more robust to different scenarios), the performance of the model reached an accuracy of 99.97%, reflecting the significant effect of data enhancement in improving the robustness and accuracy of the model. These results demonstrate the significant potential of the present invention in improving the accuracy and robustness of zero-day malicious code detection, and provide a more efficient and accurate solution for zero-day malicious code detection in complex environments.

[0199] from Figure 8 From the confusion matrix, we can see that the detection accuracy of the model of this application has been significantly improved in all categories, which further verifies the superiority of TransDiffusionNet in the zero-day malicious code detection task; the model effectively alleviates the confusion problem between categories, especially in accurately distinguishing difficult-to-identify malicious code samples; at the same time, through more precise feature extraction and fusion strategies, TransDiffusionNet shows strong generalization and robustness, which not only improves the recognition accuracy of complex malicious code samples, but also enhances the ability to distinguish minority class samples. This result fully proves the reliability and stability of the model constructed by this application in dealing with actual application scenarios.

[0200] (10) Ablation experiment

[0201] In order to better evaluate the improved TransDiffusionNet model, some key components and design choices in the TransDiffusionNet model are gradually removed or modified to evaluate their impact on model performance. Therefore, in the ablation experiment, the TransXNet backbone network, visualization method (Visualization), normalization method (Normalization), EMA attention mechanism, Focal Loss loss function and DDIM diffusion operation are controlled respectively. The results of the TransDiffusionNet ablation experiment are shown in Table 11.

[0202] Table 11 Ablation experiment table

[0203]

[0204] From the ablation experiment in Table 11, it can be seen that after improving the malicious code visualization method alone, the model accuracy increased by 0.74 percentage points, indicating the positive impact of information integrity and feature diversity on model performance. After improving the normalization method, the accuracy increased from 98.06% to 98.39%, solving the problem of texture feature changes during the normalization process, avoiding model misjudgment, and further improving detection accuracy.

[0205] After the introduction of the EMA module, the accuracy rate increased by 0.46 percentage points, indicating that EMA enhances the model's multi-scale feature integration and context perception capabilities. Although the ablation experiment of the Focal Loss loss function has a small overall improvement, it has improved the model's accuracy, verifying its effectiveness in improving model performance.

[0206] The ablation experiment of DDIM shows the importance of this method in TransDiffusionNet. After introducing DDIM, the accuracy rate increased by 0.91 percentage points. The generated samples simulated zero-day malicious code, and after combining with the original samples, the detection accuracy and robustness of the model were significantly improved.

[0207] At the same time, ablation experiments also verified the effectiveness of each module. These improvements not only ensure the integrity of information and the diversity of features, avoid the change of texture features during normalization, but also enhance the integration capability of multi-scale features. In addition, by generating high-quality samples, the detection efficiency and accuracy of zero-day malicious code are further improved.

Claims

1. A method for constructing a zero-day malicious code detection model, characterized in that: Taking TransXNet as the basic model, training and optimizing it can obtain the zero-day malicious code detection model. The method specifically includes: Preprocessing the malicious code sample data to obtain training set data; and The TransXNet is trained and optimized using the training set data to obtain the detection model; The preprocessing of the malicious code sample data includes: using a diffusion model DDIM to perform a diffusion operation on the visualized malicious code image to generate a new malicious code image, and the new malicious code image and the original malicious code image together constitute the training set data.

2. The method for constructing a zero-day malicious code detection model according to claim 1, characterized in that: The diffusion model DDIM is used to perform a diffusion operation on the visualized malicious code image to generate a new malicious code image, specifically including: adding noise to the data sample to gradually turn it into pure noise, and then gradually restoring the details and structure of the image through the inverse denoising process obtained through training, thereby generating a new complex zero-day malicious code image.

3. The method for constructing a zero-day malicious code detection model according to claim 1, characterized in that: The preprocessing of the malicious code sample data also includes: Performing visualization processing on the malicious code to obtain a visualization image of the malicious code; And / or further perform image size normalization processing on the malicious code visualization image.

4. The method for constructing a zero-day malicious code detection model according to claim 3, characterized in that: The image size normalization process is performed using a bicubic interpolation algorithm and a sharpening technique, and / or the malicious code is visualized, including: Convert the binary file of malicious code into grayscale image; Convert assembly code files to RGB images; By fusing the grayscale image with the RGB image, we can get a visualization image of the malicious code.

5. The method for constructing a zero-day malicious code detection model according to any one of claims 1 to 4, characterized in that: The calculation formula of the loss function Focal Loss of the TransXNet model is: FL(p t )=-a t (1-p t ) γ log(p t ) Among them, p t is the model’s predicted probability for the true category, α t is a balancing parameter used to balance the ratio of positive and negative samples, and γ is an adjustment parameter used to adjust the weight of difficult and easy samples; And / or the TransXNet adopts an improved TransXNet model, which includes: Multi-scale Attention Module (EMA), which is used to extract features from the input sequence through multi-scale convolution and pyramid pooling, and perform feature fusion through the cross-spatial attention mechanism; Dynamic position encoding module DPE, used to dynamically generate position information of input sequence; A lightweight dual dynamic token mixer D-Mixer, which is used to extract global and local dynamic information from the fused features; and The multi-scale feed-forward network MS-FFN is used to perform further cross-channel and multi-scale feature aggregation and pass the result to the output layer.

6. The method for constructing a zero-day malicious code detection model according to claim 5, characterized in that: The multi-scale attention module EMA includes two parallel sub-networks, which extract and fuse features of the input sequence, specifically including: Divide the input feature map X into multiple sub-feature groups; The two sub-networks respectively process the sub-feature groups using different convolution kernels to obtain output feature maps of the two sub-networks; The output feature maps of the two sub-networks are then fused through the cross-spatial attention mechanism.

7. A zero-day malicious code detection model construction device, characterized in that: include: A training set data preprocessing unit, which is used to preprocess the malicious code sample data to obtain training set data; as well as A model training optimization unit, which is used to train and optimize TransXNet using the preprocessed training set data to obtain the detection model; The training set data preprocessing unit includes a diffusion module, which is used to perform a diffusion operation on the visualized malicious code image using a diffusion model DDIM to generate a new malicious code image, and the new malicious code image and the original malicious code image together constitute the training set data.

8. A zero-day malicious code detection model, characterized in that: The model is constructed by the method described in any one of claims 1 to 6.

9. A zero-day malicious code detection method, characterized in that: include: Preprocess the zero-day malicious code to be detected to obtain a visualized malicious code image; The visualized malicious code image is input into the zero-day malicious code detection model in claim 8, or constructed by any method in claims 1-6, to classify and detect the malicious code image to obtain a zero-day malicious code detection result.

10. The zero-day malicious code detection method according to claim 9, characterized in that: The preprocessing of the zero-day malicious code to be detected includes malicious code visualization processing and / or image size normalization processing, wherein: The visualization processing specifically includes: Convert the binary file of malicious code into grayscale image; Convert assembly code files to RGB images; The grayscale image is merged with the RGB image to obtain a visualization image of the malicious code; The image size normalization process is performed using a bicubic interpolation algorithm and a sharpening technique.

Images