A zero-day malicious code detection model, method, and model construction device and method

Abstract

The present invention discloses a zero-day malicious code detection model, method, and model construction device and method, which belong to the technical field of malicious code detection. The model construction method of the present invention uses TransXNet as the basic model and uses a training set to train and optimize it to obtain the zero-day malicious code detection model, wherein the diffusion model DDIM is used to perform a diffusion operation on the visualized malicious code image to generate a new malicious code image, and the new malicious code image is combined with the original malicious code image to form the training set data. The construction method of the present invention not only improves the model's detection accuracy and robustness for zero-day malicious code, but also improves the problem of data set imbalance, thereby providing an effective technical solution to solve the problems existing in existing zero-day malicious code detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of malicious code detection, and more specifically, relates to a zero-day malicious code detection model, method, and model construction device and method. Background Art

[0002] In today's information age, computers and the internet have become key pillars of society. With the acceleration of digitalization, cybersecurity threats are becoming increasingly severe. Malicious code, a primary tool in cyberattacks, poses a significant threat to the security of individuals, businesses, and even nations. Malicious code is software or scripts designed to damage systems, steal information, or perform other malicious operations. It can take various forms, including viruses, worms, and Trojan horses.

[0003] Zero-day malware, which exploits undisclosed or unpatched vulnerabilities, is highly concealed and destructive, posing a significant challenge to cybersecurity. Victims are often unprepared until the vulnerability is exposed or patched, leaving them vulnerable to attack. As malicious code and cyberattack methods continue to evolve, traditional security measures are increasingly unable to cope with this challenging situation. Developing more advanced malicious code detection technologies to improve accuracy and real-time detection has become a pressing research topic in cybersecurity.

[0004] Due to the unknown and hidden nature of zero-day malware, traditional detection methods face significant challenges. Signature-based detection methods rely on known features and are ineffective against unknown zero-day malware. While behavioral analysis and anomaly detection methods can identify some anomalous behaviors, they remain inadequate against advanced zero-day attacks. In recent years, deep learning technology, with its ability to automatically extract complex features, has demonstrated significant potential in zero-day malware detection, providing a more efficient means of detection.

[0005] Current malicious code detection technologies mainly include the following categories:

[0006] (1) Signature-based detection method: Rapid detection is achieved by matching the signatures of known malicious code. This type of method is widely used in malicious code databases. Its advantages are fast detection speed and low resource consumption. However, it has obvious drawbacks. It mainly relies on known feature libraries and cannot effectively identify zero-day malicious code or new variants of malware. It is difficult to cope with the challenges brought by the continuous evolution of attack methods. Since zero-day malicious code usually exploits unknown vulnerabilities, the victim system is defenseless until the vulnerability is fixed. This severely limits the practical application of signature-based detection methods.

[0007] (2) Byte sequence-based detection methods: Deep learning models, such as convolutional neural networks (CNNs) or long short-term memory networks (LSTMs), are used to analyze the binary byte streams of malicious code. This type of method improves the generalization ability of malicious code detection to a certain extent by automatically extracting patterns and features from the byte stream. However, due to the varying lengths and sparse features of malicious code byte streams, the detection accuracy is insufficient, especially when dealing with complex malware types, the generalization ability of the model is often weak. This is because binary data lacks structural features in form, making it difficult for the model to capture global patterns.

[0008] (3) Detection methods based on behavioral analysis: By monitoring the dynamic behavior of malicious code in a sandbox environment, the dynamic behavior of malicious code is analyzed. This type of method has high detection accuracy in specific scenarios and can provide a comprehensive detection perspective through behavioral data. However, its main problem is that it relies on a virtual execution environment, which results in high resource consumption. At the same time, some advanced malicious code has the ability to circumvent sandbox technology, making this type of method easy to bypass. In addition, behavioral analysis methods are difficult to provide effective detection for malicious behaviors that have not been executed.

[0009] (4) Detection methods based on generative adversarial networks (GANs): Generate realistic malware samples to simulate zero-day malicious code, thereby improving the detection system's ability to identify unknown threats. Although GANs perform well in generating malicious samples and enhancing data, their training process is often unstable, prone to mode collapse, and extremely sensitive to hyperparameters, lacking a clear probabilistic interpretation. In addition, the adversarial training between the generator and the discriminator also increases the complexity of the model and the difficulty of application.

[0010] In summary, while the aforementioned existing detection methods have made significant progress in malware detection, they still exhibit significant limitations when dealing with the unknowns and complexity of zero-day malware. Key issues include insufficient detection capabilities for unknown threats, weak model generalization, unstable training processes, and dependence on the execution environment. These issues stem primarily from limitations in feature extraction, data sparsity processing, and dynamic behavior monitoring. Technological innovation is urgently needed to improve the accuracy, robustness, and real-time performance of malware detection. Summary of the Invention

[0011] The purpose of the present invention is to provide a zero-day malicious code detection model, method, and model construction device and method to solve the problem that the accuracy, robustness, and real-time performance of zero-day malicious code detection cannot be guaranteed due to the limitations of existing detection methods in feature extraction, data sparsity processing, or dynamic behavior monitoring.

[0012] In order to achieve the above object, the technical solution provided by the present invention is:

[0013] A first aspect of the present invention provides a method for constructing a zero-day malicious code detection model. The method uses TransXNet as a basic model and trains and optimizes it to obtain the zero-day malicious code detection model. The method specifically includes:

[0014] Preprocessing the malicious code sample data to obtain training set data; and

[0015] Using the training set data to train and optimize TransXNet, the detection model is obtained;

[0016] The preprocessing of the malicious code sample data includes: performing a diffusion operation on the visualized malicious code image using a diffusion model DDIM to generate a new malicious code image, and the new malicious code image and the original malicious code image together constitute the training set data.

[0017] In order to address the shortcomings of traditional methods in zero-day malicious code detection, the present invention uses TransXNet as the basic model and utilizes the diffusion model DDIM to generate high-quality malicious code images. The generated new malicious code images are combined with the original malicious code images to form a training set data for training TransXNet. This not only improves the model's detection accuracy and robustness for zero-day malicious code, but also improves the problem of data set imbalance, thereby providing an effective technical solution to solve the problems existing in existing zero-day malicious code detection.

[0018] According to the construction method of any technical solution described in the first aspect of the present invention, the diffusion operation of the visualized malicious code image using the diffusion model DDIM to generate a new malicious code image specifically includes: adding noise to the data sample to gradually convert it into pure noise, and then gradually restoring the details and structure of the image through the inverse denoising process obtained through training, thereby generating a new complex zero-day malicious code image.

[0019] According to the construction method of any technical solution described in the first aspect of the present invention, the preprocessing of the malicious code sample data further includes:

[0020] Performing visualization processing on the malicious code to obtain a visualization image of the malicious code;

[0021] And / or further perform image size normalization processing on the malicious code visualization image.

[0022] According to the construction method of any technical solution described in the first aspect of the present invention, the image size normalization processing is performed using a bicubic interpolation algorithm and a sharpening technique, and / or the malicious code is visualized, including:

[0023] Convert the binary file of malicious code into grayscale image;

[0024] Convert assembly code files to RGB images;

[0025] By fusing the grayscale image with the RGB image, we can obtain a visualization image of the malicious code.

[0026] To ensure the integrity and feature diversity of malicious code information, the present invention adopts an improved malicious code visualization method, which fuses the grayscale image generated by the .bytes file with the RGB image generated by the opcode feature map extracted from the .asm file. By integrating information from different sources, the diversity of features is significantly improved and the comprehensiveness of feature extraction is enhanced, providing a solid foundation for efficient and accurate detection.

[0027] By combining the bicubic interpolation algorithm with sharpening technology to further normalize the size of the malicious code visualization image, not only does it ensure that the texture features remain unchanged during the normalization process, but it also improves the image clarity, making the normalized image more suitable for subsequent malicious code detection tasks. This can effectively solve the problem of texture feature changes that may be caused by the normalization process of the malicious code image.

[0028] According to the construction method of any technical solution described in the first aspect of the present invention, the calculation formula of the loss function Focal Loss of the TransXNet model is:

[0029] FL(p t )=-α t (1-p t ) γ log(p t )

[0030] Among them, p t is the model’s predicted probability for the true category, α t is a balancing parameter used to balance the ratio of positive and negative samples, and γ is an adjustment parameter used to adjust the weight of difficult and easy samples.

[0031] The present invention further improves the cross-entropy loss function of the original model and adopts Focal Loss as the loss function. Focal Loss significantly improves the classification performance of the model under data imbalance conditions by reducing the influence of easy-to-classify samples and increasing the weight of difficult-to-classify samples.

[0032] According to the construction method of any technical solution described in the first aspect of the present invention, the TransXNet adopts an improved TransXNet model, which includes:

[0033] Multi-scale Attention Module (EMA), which extracts features from the input sequence through multi-scale convolution and pyramid pooling, and fuses features through a cross-spatial attention mechanism;

[0034] Dynamic position encoding module DPE, used to dynamically generate position information of the input sequence;

[0035] A lightweight dual dynamic token mixer (D-Mixer) for extracting global and local dynamic information from the fused features; and

[0036] Multi-scale feedforward network MS-FFN is used to introduce multi-scale features so that the model can combine information of different scales during feature extraction.

[0037] EMA blocks the input image and embeds it into a high-dimensional space, providing a foundation for subsequent feature extraction. It then outputs the embedded features for subsequent processing by the Dynamic Position Encoding (DPE) and Feature Mixing Module (D-Mixer). The DPE injects positional information into the input features, enhancing spatial relationship modeling capabilities. It is located at the beginning of each stage, and the processed features are passed to the Feature Mixing Module (D-Mixer). During feature processing, the D-Mixer continuously integrates contextual information and passes the results to the Multi-Scale Feedforward Network (MS-FFN). The MS-FFN receives the features from the D-Mixer, performs further cross-channel and multi-scale feature aggregation, and then passes the results to the output layer.

[0038] As a preferred implementation, to reduce model computational overhead and improve detection efficiency, this paper introduces a multi-scale attention module (EMA) based on the existing TransXNet model. EMA improves model performance by enhancing multi-scale feature integration, context awareness, and noise suppression capabilities. Furthermore, the EMA module effectively captures features at different scales, enhancing the ability to recognize diverse features and accurately detecting malicious code in complex scenarios while reducing computational overhead.

[0039] According to the construction method of any technical solution described in the first aspect of the present invention, the multi-scale attention module EMA includes two parallel sub-networks, which extract and fuse features of the input sequence, specifically including:

[0040] Divide the input feature map X into multiple sub-feature groups;

[0041] The two sub-networks respectively process the sub-feature groups using different convolution kernels to obtain output feature maps of the two sub-networks;

[0042] The output feature maps of the two sub-networks are then fused through the cross-spatial attention mechanism.

[0043] A second aspect of the present invention provides a zero-day malicious code detection model construction device, comprising:

[0044] a training set data preprocessing unit, which is used to preprocess the malicious code sample data to obtain training set data; and

[0045] A model training optimization unit, which is used to train and optimize TransXNet using the preprocessed training set data to obtain the detection model;

[0046] The training set data preprocessing unit includes a diffusion module, which is used to perform a diffusion operation on the visualized malicious code image using a diffusion model DDIM to generate a new malicious code image. The new malicious code image and the original malicious code image together constitute the training set data.

[0047] According to the device of any technical solution described in the second aspect of the present invention, the diffusion operation of the visualized malicious code image using the diffusion model DDIM to generate a new malicious code image specifically includes: adding noise to the data sample to gradually convert it into pure noise, and then gradually restoring the details and structure of the image through the trained inverse denoising process, thereby generating a new complex zero-day malicious code image.

[0048] A third aspect of the present invention provides a zero-day malicious code detection model, which is constructed using any of the technical solutions described in the first aspect of the present invention.

[0049] A fourth aspect of the present invention provides another zero-day malicious code detection model, which adopts an improved TransXNet model and includes:

[0050] Multi-scale Attention Module (EMA), which extracts features from the input sequence through multi-scale convolution and pyramid pooling, and fuses features through a cross-spatial attention mechanism;

[0051] Dynamic position encoding module DPE, used to dynamically generate position information of the input sequence;

[0052] A lightweight dual dynamic token mixer (D-Mixer) for extracting global and local dynamic information from the fused features; and

[0053] Multi-scale feedforward network MS-FFN is used to introduce multi-scale features so that the model can combine information of different scales during feature extraction.

[0054] By improving the existing TransXNet model and introducing the multi-scale attention module EMA, we can effectively capture features of different scales, enhance the ability to recognize diverse features, and accurately detect malicious code in complex scenarios while reducing computational overhead.

[0055] According to the detection model of any technical solution described in the fourth aspect of the present invention, the multi-scale attention module EMA includes two parallel sub-networks, which extract and fuse features of the input sequence, specifically including:

[0056] Divide the input feature map X into multiple sub-feature groups;

[0057] The two sub-networks respectively process the sub-feature groups using different convolution kernels to obtain output feature maps of the two sub-networks;

[0058] The output feature maps of the two sub-networks are then fused through the cross-spatial attention mechanism.

[0059] A fifth aspect of the present invention provides a zero-day malicious code detection method, comprising:

[0060] Pre-process the zero-day malicious code to be detected to obtain a visual malicious code image;

[0061] The visualized malicious code image is input into any zero-day malicious code detection model described in the third or fourth aspect of the present invention, or constructed using the method described in the first aspect, to classify and detect the malicious code image and obtain a zero-day malicious code detection result.

[0062] According to the zero-day malicious code detection method of any technical solution described in the fifth aspect of the present invention, the pre-processing of the zero-day malicious code to be detected includes malicious code visualization processing and / or image size normalization processing, wherein:

[0063] The visualization processing specifically includes:

[0064] Convert the binary file of malicious code into grayscale image;

[0065] Convert assembly code files to RGB images;

[0066] Fusion of grayscale image and RGB image to obtain malicious code visualization image;

[0067] The image size normalization process is performed using a bicubic interpolation algorithm and a sharpening technique.

[0068] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0069] (1) This invention uses TransXNet as its base model. During model training and optimization, a diffusion operation is performed on the visualized sample images to generate new malicious code image samples, which are then combined with the original samples for model training. This method effectively simulates unknown zero-day malicious code, significantly improving the accuracy and robustness of the detection model in zero-day malicious code detection, and addressing the shortcomings of existing technologies in processing unknown attack samples.

[0070] (2) By fusing the grayscale image generated by the .bytes file with the RGB image generated by the opcode mapping of the .asm file, the present invention significantly improves the information integrity and feature diversity during the malware visualization process. Compared with traditional methods, the present invention excels in the comprehensiveness of malware feature extraction, can capture richer feature information, and improves the precision and accuracy of detection.

[0071] (3) The present invention combines the bicubic interpolation algorithm with the sharpening technology to perform size normalization on the malicious code visualization image, thereby ensuring the retention of texture features during the normalization process and improving the image clarity. This improvement enables the malicious code detection model to more accurately identify malicious code features, ensure the quality of the image after normalization, and effectively solve the problem of texture feature changes in the malicious code image during the normalization process in the prior art.

[0072] (4) This paper further improves the TransXNet model by introducing a multi-scale attention module (EMA). By enhancing the integration of multi-scale features and contextual awareness, it effectively improves the model's ability to identify malicious code in complex scenarios. Compared to existing technologies, this paper not only outperforms in diverse feature recognition but also reduces computational overhead, significantly improving detection efficiency.

[0073] (5) The present invention further adopts Focal Loss as an improved loss function. Focal Loss can effectively reduce the loss of easy-to-classify samples and focus on difficult-to-classify samples, thereby improving the performance of the model on unbalanced datasets and enhancing the classification accuracy in malicious code detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A visualization flow chart of malicious code according to an embodiment of the present invention;

[0075] Figure 2 A comparison chart of different normalization processes for malicious code visualization images;

[0076] Figure 3 This is a schematic diagram of the EMA module structure according to an embodiment of the present invention;

[0077] Figure 4 This is a diagram of the improved TransXNet structure in an embodiment of the present invention;

[0078] Figure 5 Schematic diagram of the construction process of TransDiffusionNet in an embodiment of the present invention;

[0079] Figure 6 Schematic diagram of the Cross Entropy Loss confusion matrix of an embodiment of the present invention;

[0080] Figure 7 Schematic diagram of the Focal Loss confusion matrix of an embodiment of the present invention;

[0081] Figure 8 Schematic diagram of a confusion matrix for a zero-day malicious code detection experiment according to an embodiment of the present invention. DETAILED DESCRIPTION

[0082] In order to further understand the content of the present invention, the present invention is now described in detail with reference to specific embodiments.

[0083] Combine Figure 5 As shown, an embodiment of the present invention provides a method for constructing a zero-day malicious code detection model. The zero-day malicious code detection model is obtained by training and optimizing TransXNet as a basic model. The method specifically includes:

[0084] Preprocessing the malicious code sample data to obtain training set data; and

[0085] Using the training set data to train and optimize TransXNet, the detection model is obtained;

[0086] The preprocessing of the malicious code sample data includes: performing a diffusion operation on the visualized malicious code image using a diffusion model DDIM to generate a new malicious code image, and the new malicious code image and the original malicious code image together constitute the training set data.

[0087] The diffusion model, by defining a forward noise addition process and a reverse denoising generation process, is able to generate high-quality and diverse samples. Compared with traditional methods, the diffusion model demonstrates significant advantages in terms of generation stability, sample diversity, and robustness of the training process.

[0088] The core idea of ​​the diffusion model is to gradually add noise to data samples until the samples become completely pure noise, and then use the learned inverse process to gradually remove the noise and restore the original data. The entire process is divided into two stages: forward diffusion and reverse generation. In the forward diffusion stage, the structure and characteristics of the data samples are gradually covered by noise, eventually turning into pure noise; in the reverse generation stage, the model gradually restores the original characteristics of the data through denoising, generating high-quality samples from the noise. This two-stage mechanism ensures the high quality and diversity of the generated samples, while significantly improving the stability and reliability of model training. The specific process is as follows:

[0089] The forward diffusion process is a process of gradually adding noise. Given a data sample a0, the forward process generates a series of samples a1, a2, a3, ..., a t , where a small amount of Gaussian noise is added at each step.

[0090]

[0091] Among them, β t is a predefined noise scheduling parameter, represents normal distribution, Ι represents the unit matrix, q(a t |a t-1 ) represents the forward process transition probability.

[0092] The reverse generation process is a step-by-step denoising process, aiming to generate t Restore to the original data b0. This process is done through a parameterized model p θ (b t-1 |b t ) to achieve it.

[0093] p θ (b t-1 ∣b t )=N(b t-1 ;μ θ (b t ,t),Σ θ (b t ,t))

[0094] Among them, θ is the parameter of the model, μ θ (b t ,t) and ∑ θ (b t ,t) are the mean and covariance of the neural network predictions, Indicates normal distribution. p θ (b t-1 |t) is the sample b at the given t-th step t Under the condition of , generate the t-1 step sample b t-1The conditional probability distribution of .

[0095] DDIM is an improved diffusion model that accelerates the data generation process by introducing a new deterministic sampling process. Compared with traditional diffusion models, DDIM significantly reduces the generation time while maintaining the generation quality. The core formula of the DDIM inverse process is as follows:

[0096]

[0097] Among them, x t is the variable at step t, α t and α t-1 are the scaling factors at step t and t-1 respectively. ∈ θ (x t ,t) is the noise predicted by the neural network, σ t is a noise-related parameter that controls the noise level at each step.

[0098] DDIM generates images through step-by-step inference. Starting from the initial time step, the uncertainty of random variables is gradually reduced by continuously iterating the dynamic system model, thereby generating high-quality images more efficiently. The present invention generates new malicious code images by applying the DDIM diffusion operation to the visualized image. This process effectively simulates unknown malicious code samples and combines the generated samples with the original samples for training the malicious code detection model TransXNet. By introducing simulated malicious code samples, the model performs better in detecting and classifying zero-day malicious code, significantly improving accuracy and robustness. In addition, the generated new samples help balance the data set, reduce the impact of the imbalance between majority and minority class samples, and provide an effective solution for zero-day malicious code detection.

[0099] In some embodiments, the pre-processing of the malicious code sample data further includes: visualizing the malicious code to obtain a malicious code visual image; combining Figure 1 , the visualization process preferably includes the following steps:

[0100] Convert the binary file (.bytes file) of the malicious code into a grayscale image;

[0101] Convert assembly code files to RGB images;

[0102] By fusing the grayscale image with the RGB image, we can obtain a visualization image of the malicious code.

[0103] Among them, the specific method of converting the binary file (.bytes file) of the malicious code into a grayscale image is further preferably: reading each 8-bit unsigned integer in groups and converting it into a decimal integer. Since the malicious code files vary in size, the traditional method of determining the image size by file size will result in information loss. To this end, the present application calculates the length and width of the image based on the number of file bytes to ensure that the size is sufficient to accommodate all the data. Subsequently, the data is converted into a two-dimensional array, and each element corresponds to a grayscale value to generate a grayscale image.

[0104] The method of converting an assembly code file (.asm file) into an RGB image specifically includes: parsing the assembly file line by line, extracting opcodes and counting their frequencies to generate an opcode frequency feature vector of length 256; normalizing the frequencies to a range of 0 to 255; and padding the vector length with zeros to form a three-channel array if it is not a multiple of 3; and finally, constructing an RGB array and generating an RGB image.

[0105] The fusion of grayscale and RGB images involves converting the grayscale image to RGBA format, with the grayscale values ​​serving as transparency information for the alpha channel. The RGB image is then resized to the same size as the grayscale image to ensure proper alignment. The RGB image is then combined with the grayscale image with transparency information to generate a fused image. This fusion combines the structural information of the grayscale image with the semantic information of the RGB image, providing richer features.

[0106] The fusion of grayscale and RGB images not only integrates information from different sources but also improves the model's recognition capabilities. This is particularly true when dealing with obfuscated, mutated, and packed malware, enabling it to better capture key features and reduce the probability of false positives and false negatives. Furthermore, this approach enriches sample diversity, mitigates overfitting, and improves the model's generalization capabilities on unseen data.

[0107] In some embodiments, the pre-processing of the malicious code sample data further includes: further normalizing the size of the malicious code visualization image. The image size normalization is further performed using a bicubic interpolation algorithm and a sharpening technique.

[0108] Because each malware file has a different number of bytes, the size of the visualized image also varies. Deep learning models typically require fixed-size input images. Therefore, image size variations can lead to inconsistent feature map sizes, complicating model design and training, impacting memory allocation and computational efficiency, and potentially hindering training. Therefore, normalizing the visualized malware images is essential.

[0109] Malicious code images contain numerous important texture features, which are crucial for models to effectively extract and identify key information within the malicious code. While malicious code often evades traditional detection methods through variants or modifications, its fundamental texture features generally remain consistent. Therefore, accurately extracting and analyzing these texture features is crucial for combating malicious code variants. To preserve the original texture features of the normalized malicious code image as much as possible, traditional methods typically use bilinear interpolation to resize the image. However, experiments have shown that even with bilinear interpolation, some texture features may still be lost or altered. To address this, embodiments of the present invention propose a method that combines a bicubic interpolation algorithm with a sharpening technique. The bicubic interpolation algorithm effectively preserves details and edge information during scaling, while the sharpening technique further enhances image clarity. This method not only effectively addresses the issue of texture feature variations during the normalization process, but also makes the normalized image clearer, highlighting key textures, thereby improving the accuracy and robustness of the model in detecting and classifying malicious code.

[0110] like Figure 2 The following are comparisons of different normalization processes on the malicious code visualization image, where (A) is the original image, (B) is the image normalized using the bilinear interpolation algorithm, and (C) is the image generated using the normalization method of the embodiment of the present invention. Figure 2 It can be clearly seen that the texture features of the malicious code image in Figure (B) have changed significantly, which may cause the detection model to learn inaccurate features, affecting the accuracy of malicious code detection. In contrast, Figure (C) maintains the texture features of the original image, providing the model with more reliable feature input, thereby improving detection accuracy and model robustness.

[0111] In some embodiments, the TransXNet model directly adopts the existing TransXNet model. TransXNet combines the local feature extraction advantages of CNN and the global feature modeling capabilities of Transformer. It can efficiently extract features and deeply understand semantics in malicious code detection, thereby significantly improving detection performance and applicability.

[0112] Due to its static characteristics, traditional convolution is difficult to dynamically adapt to input changes, resulting in a difference in representational capabilities between convolution and self-attention. To this end, TransXNet introduces a lightweight dual dynamic token mixer (D-Mixer), which combines the global self-attention mechanism (Overlapping Spatial Reduction Attention, OSRA) and input-dependent deep convolution (Input-dependent Depthwise Convolution, IDConv). While maintaining input dependency, D-Mixer utilizes a wide receptive field and inductive bias to powerfully capture global and local dynamic information. Specifically, D-Mixer divides the input feature map into two parts: one part is processed by the global self-attention module, and the other part is processed by input-dependent deep convolution. This dual processing mechanism significantly reduces computational overhead while enhancing the generalization ability of the model and ensures that performance is effectively improved.

[0113] The Dynamic Position Encoding (DPE) module is a key mechanism in TransXNet for dynamically generating position information in input sequences. Traditional Transformer models use predefined sine and cosine functions as static position encodings, which cannot be adjusted based on the specific input. DPE, on the other hand, dynamically generates position encodings during model runtime, better capturing the complex relationships and structural changes in the input sequence.

[0114] The Multi-scale Feed-forward Network (MS-FFN) introduces a multi-scale feature processing mechanism, enabling the model to combine information at different scales during feature extraction. Compared to traditional feed-forward networks, MS-FFN can more comprehensively represent the complexity of input data, making the model more robust when dealing with interference such as deformation and noise, thereby improving the model's robustness and expressiveness.

[0115] Overall, TransXNet, through its innovative module design, not only enhances the model's feature extraction capabilities, but also significantly optimizes the model's computational efficiency and accuracy, enabling it to perform well in malicious code detection tasks.

[0116] In order to effectively integrate features of different scales, improve the model's ability to capture various features, and enhance the perception of contextual information so that the model can still accurately identify malicious code in complex backgrounds, the TransXNet model is optimized in some embodiments, that is, a multi-scale attention module EMA is introduced on the basis of the above-mentioned existing TransXNet model, and features of different scales are integrated in combination with a cross-space learning mechanism. The EMA module extracts features through multi-scale convolution and pyramid pooling, and then fuses them through a cross-space attention mechanism, which not only captures details but also understands the global context. In addition, the EMA module has noise suppression capabilities, which improves classification performance in complex backgrounds, especially when dealing with variant, obfuscated and packed malicious codes.

[0117] To reduce computational complexity, the EMA module reshapes some channels into batch dimensions and groups the channel dimensions into multiple sub-features. Specifically, Figure 3 As shown, the EMA module groups the input feature map X into G sub-features, each sub-feature X i Contains several channels.

[0118] X=[X1,X2,…,X G ]

[0119] The EMA module consists of two parallel sub-networks, one sub-network uses a 1×1 convolution kernel to process features, and the other sub-network uses a 3×3 convolution kernel to process features. The output feature maps of each sub-network are represented as Y1 and Y2 respectively.

[0120] Y1=Conv 1×1 (X)

[0121] Y2=Conv 3×3 (X)

[0122] In each parallel sub-network, the EMA module captures pixel-level pairwise relationships through cross-dimensional interactions. Specifically, it encodes global information in each sub-feature map, recalibrates channel weights, and fuses the output features of the two parallel sub-networks. Through a cross-space learning approach, the EMA module achieves local cross-channel interactions without reducing the channel dimension. This module's multi-scale parallel design effectively enhances the model's contextual awareness, enabling it to better understand the relationships between different regions in an image, thereby improving the accuracy and robustness of image classification tasks.

[0123] In the TransXNet model after applying the EMA module, the input feature map is divided into multiple sub-feature groups, and these sub-features are processed by different convolution kernels to achieve effective integration of multi-scale information. This mechanism significantly enhances the context perception ability of the model, thereby improving the accuracy and robustness of classification. In various image classification tasks, the EMA module not only greatly reduces the computational complexity, but also improves the detection efficiency, especially in complex malicious code identification tasks, showing excellent performance and adaptability. The TransXNet model after applying the EMA module is as follows Figure 4 shown.

[0124] In order to further eliminate the impact of dataset imbalance on model performance, in some embodiments, the loss function of the TransXNet model is further optimized. The calculation process of the loss function Focal Loss is shown in the following formula:

[0125] FL(p t )=-α t (1-p t ) γ log(p t )

[0126] Among them, p t is the model’s predicted probability for the true category, α t is a balancing parameter used to balance the ratio of positive and negative samples, and γ is an adjustment parameter used to adjust the weight of difficult and easy samples.

[0127] In classification tasks, positive and negative samples are often unbalanced. To solve this problem, Focal Loss introduces a balance parameter α t , weight the loss of positive and negative samples with different weights. If the ratio of positive and negative samples is seriously unbalanced, you can set α t By adjusting the value of γ, the weight distribution of easy and hard examples can be controlled. When γ = 0, Focal Loss degenerates into a standard cross-entropy loss. As γ increases, the loss weight of easy examples gradually decreases, causing the model to pay more attention to hard examples. Through this mechanism, Focal Loss effectively improves the model's performance on imbalanced datasets, especially in the classification accuracy of minority class examples.

[0128] As a further preferred embodiment of any embodiment of the present invention, the calculation formula of the loss function FocalLoss of the TransXNet model is:

[0129] FL(p t )=-α t (1-p t )γ log(p t )

[0130] Among them, p t is the model’s predicted probability for the true category, α t is a balancing parameter used to balance the ratio of positive and negative samples, and γ is an adjustment parameter used to adjust the weight of difficult and easy samples.

[0131] An embodiment of the present invention further provides a device for constructing a zero-day malicious code detection model, comprising:

[0132] a training set data preprocessing unit, which is used to preprocess the malicious code sample data to obtain training set data; and

[0133] A model training optimization unit, which is used to train and optimize TransXNet using the preprocessed training set data to obtain the detection model;

[0134] The training set data preprocessing unit includes a diffusion module, which is used to perform a diffusion operation on the visualized malicious code image using a diffusion model DDIM to generate a new malicious code image. The new malicious code image and the original malicious code image together constitute the training set data.

[0135] As a further preferred embodiment, the training set data preprocessing unit also includes a visualization processing module and / or an image size normalization processing module, wherein the specific processing method of the visualization processing module and / or the image size normalization processing module is preferably to adopt any of the visualization and normalization processing methods described above, which will not be introduced in detail here.

[0136] It should be noted that the above-mentioned TransXNet model can directly adopt the existing TransXNet model, or can adopt the improved TransXNet model in the present invention, which will not be introduced in detail here.

[0137] An embodiment of the present invention further provides a zero-day malicious code detection model, which is constructed using the model construction method of any of the above embodiments of the present invention.

[0138] An embodiment of the present invention further provides a method for detecting zero-day malicious code, including:

[0139] Pre-process the zero-day malicious code to be detected to obtain a visual malicious code image;

[0140] The visualized malicious code image is input into any of the above embodiments, or a zero-day malicious code detection model is constructed using the construction method in any of the above embodiments to classify and detect the malicious code image and obtain a zero-day malicious code detection result.

[0141] As a preferred implementation method, the preprocessing of the zero-day malicious code to be detected includes malicious code visualization processing and / or image size normalization processing. The malicious code visualization processing and image size normalization processing method is further preferably to adopt the visualization and normalization processing method described in the present invention, which will not be repeated here.

[0142] Model performance test experiment:

[0143] (1) Experimental environment and parameter settings

[0144] In order to evaluate the performance of the improved model (TransDiffusionNet), a deep learning-based zero-day malicious code detection platform was built. The experimental environment configuration is shown in Table 1.

[0145] Table 1 Experimental environment configuration

[0146]

[0147]

[0148] The experimental model is built based on the PyTorch framework. The core parameters and parameter values ​​during the model training process are shown in Table 2.

[0149] Table 2 Experimental parameter configuration

[0150]

[0151] (2) Dataset Introduction

[0152] In order to evaluate the performance of the model on different datasets, this application selected the Malimg dataset and the Microsoft malicious code dataset as training datasets, but other visualized or directly visualized malicious code image datasets can also be used.

[0153] The Malimg dataset is a well-known dataset specifically designed for malware classification research. It contains 9,339 images from 25 different malware families. These images are generated by converting the malware's binary code into grayscale images, with each image representing a single malware sample. The Malimg dataset is notable for its visual diversity, with different malware families exhibiting unique textures and patterns in the images. This diversity makes the dataset particularly suitable for testing models' ability to identify and distinguish different malware types. Due to the imbalance in the number of samples between families, the Malimg dataset also poses additional challenges to the model's ability to handle data skew and sample scarcity. The specific composition, family categories, and sample distribution of the Malimg dataset are shown in Table 3.

[0154] Table 3 Maling dataset

[0155]

[0156]

[0157] The Microsoft Malware Dataset, also known as the BIG 2015 dataset, is a large-scale malware classification challenge dataset released by Microsoft in 2015. The dataset includes nine malware families and a total of 10,868 samples. The dataset consists of two types of files: .byte files, representing malware binaries without a PE header; and .asm files, representing the malware instruction sets obtained through disassembly. These files provide rich structured information, enabling the generation of multi-dimensional malware images. The BIG 2015 dataset covers a wide range of malware types and also exhibits an imbalance in the number of sample families within the dataset. For example, the malware family Simda has the fewest samples, with only 42, accounting for less than 1% of the total sample count. This makes the dataset an ideal tool for evaluating model performance in complex and diverse scenarios. Table 4 shows the detailed composition, family categories, and sample distribution of the BIG 2015 dataset.

[0158] Table 4 Microsoft malicious code dataset

[0159]

[0160] The combined use of these two datasets can comprehensively examine the model's robustness, generalization ability, and ability to handle unbalanced datasets in dealing with malicious code classification tasks.

[0161] (3) Evaluation indicators

[0162] This application comprehensively evaluates the performance of the TransDiffusionNet model using four indicators: Accuracy, Precision, Recall, and F1-score. Accuracy measures the proportion of samples predicted correctly by the model, reflecting the overall classification effect of the model. Precision indicates the proportion of samples predicted as positive that are actually positive, evaluating the accuracy of the model in positive prediction. Recall indicates the proportion of samples that are actually positive that are correctly identified as positive, reflecting the detection ability of the model. The F1 score, as the harmonic mean of precision and recall, comprehensively balances the two and is suitable for evaluating classification performance under imbalanced datasets.

[0163]

[0164] Among them, TP (True Positive): the number of samples predicted to be positive and actually positive, TN (True Negative): the number of samples predicted to be negative and actually negative, FP (False Positive): the number of samples predicted to be positive but actually negative, FN (False Negative): the number of samples predicted to be negative but actually positive.

[0165] This application uses FID (Fréchet Inception Distance) as a quality metric for images generated by diffusion models. It measures the similarity between generated and real images by calculating the statistical distance between them in feature space. FID evaluates the quality and diversity of generated images by calculating the statistical distribution distance between real and generated images in the feature extraction layer of a pre-trained Inception v3 model (which converts images into high-dimensional feature representations to assess the distribution difference between generated and real images, thereby measuring the quality of generated images).

[0166]

[0167] Among them, μ R 、μ G represents the mean vector of features extracted from real data and generated data, ∑ R ,∑ G represents the covariance matrix of features extracted from real data and generated data, ‖μ R -μ G ‖ 2The square of the Euclidean distance between two mean vectors measures the difference between the generated image and the real image in feature space. The square root of the product of the two covariance matrices measures the similarity between the covariance matrices of the generated image and the real image. This formula measures the statistical distance between the generated image and the real image in feature space, that is, the difference in their distribution. Lower FID values ​​indicate better quality of the generated image and a closer distribution to the real image.

[0168] (4) Malicious code visualization improvement experiment

[0169] In order to ensure the integrity of information and diversity of features during the visualization of malicious code, improve the comprehensiveness of feature extraction, and thus enhance the performance of the model in malicious code detection tasks, this application fuses the grayscale image generated by the .bytes file in the BIG 2015 dataset with the RGB image of the opcode feature map extracted from the .asm file. Based on the TransXNet model, the experiment compares the traditional method of determining the image size based on the file size and the method proposed in this application of calculating the image size based on the number of bytes in the file content (the method before improvement in the table) to verify the impact of information integrity on model performance. In addition, the experiment also compares the grayscale image before fusion and the image after fusion (the improved image in the table is the image whose length and width are calculated based on the number of bytes in the file content, and the fused image refers to the image after the grayscale image and the RGB image are fused during visualization) to further verify the effect of feature diversity on improving model performance.

[0170] Table 5 Comparative experiments of visualization methods

[0171]

[0172] The experimental results in Table 5 demonstrate that the improved visualization method significantly improves the performance of the TransXNet model. Compared to traditional methods that determine fixed image size, the strategy of dynamically calculating image size based on file byte count not only effectively improves the model's detection performance but also optimizes the information integrity of the input data. In particular, fusing grayscale images with RGB images of opcode feature maps extracted from .asm files further enhances the accuracy and generalization of malware detection. Experimental results demonstrate that information integrity and feature diversity play a key role in improving model performance. By comprehensively utilizing multi-source features, the model not only achieves significant breakthroughs in detection accuracy but also enhances its adaptability to complex and mutating malware.

[0173] (5) Experiment on improving normalization method

[0174] To address the potential change in texture features during image normalization, this application uses the TransXNet model and the BIG 2015 dataset, employing a bicubic interpolation algorithm and sharpening techniques to normalize malware visualization images. The experiment compares the traditional bilinear interpolation algorithm with the improved normalization method to evaluate the performance improvement of the model.

[0175] Table 6 Comparative experiment of normalization methods

[0176]

[0177] Traditional bilinear interpolation algorithms, when processing malware image normalization, suffer from texture feature changes, limiting model performance. To address this issue, this application proposes a new method that combines bicubic interpolation with sharpening techniques. Experimental results show that the improved method significantly improves model performance, reaching an accuracy of 99.17%, a significant improvement over the 98.8% achieved by the traditional method. This demonstrates the effectiveness of the improved normalization method in maintaining image clarity and enhancing detection robustness, effectively addressing the negative impact of texture feature changes on model performance.

[0178] (6) Model improvement experiment

[0179] To effectively integrate features at different scales, improve the model's ability to capture diverse features, and enhance its understanding of contextual information, enabling the model to accurately identify malicious code even in complex environments, the experiment used TransXNet as the base model and introduced the Multi-Scale Attention Module (EMA). To comprehensively evaluate the impact of the Multi-Scale Attention Module on the TransXNet model, the experiment compared the performance changes before and after applying EMA.

[0180] Table 7 TransXNet model improvement experiment

[0181]

[0182] The experimental results in Table 7 show that the TransXNet model using the EMA module achieves significant improvements in all four metrics, reaching 99.26% in each, significantly outperforming the 99.17% achieved without the EMA module. This result demonstrates that the introduction of the EMA module not only enhances the model's multi-scale feature integration capabilities but also significantly improves the accuracy and stability of malware detection while reducing noise interference, thereby enhancing the model's adaptability in handling diverse and complex malware scenarios.

[0183] (7) Loss function improvement experiment

[0184] In order to eliminate the impact of dataset imbalance on model performance, the experiment improved the loss function and used the BIG2015 dataset to evaluate the performance of Focal Loss, MultiMargin Loss, and Cross Entropy Loss on the TransXNet model. The results are shown in Table 8.

[0185] Table 8 Loss function improvement experiment

[0186]

[0187] By assigning higher weights to difficult-to-classify samples, Focal Loss mitigates the impact of dataset imbalance on model training, effectively improving the model's ability to detect malicious code. Experimental results show that the introduction of Focal Loss enables the model to better handle difficult-to-classify samples, further improving detection accuracy and comprehensiveness. This result validates the effectiveness of Focal Loss in improving model performance, particularly when working with imbalanced datasets.

[0188] By comparison Figure 6 and Figure 7 As can be seen from the confusion matrix in Figure 2, after using the Focal Loss loss function, the detection accuracy of Simda, the smallest malicious code family in the BIG2015 dataset, increased by 13%. This shows that Focal Loss can effectively enhance the model's ability to recognize small sample categories when dealing with class imbalanced datasets, further verifying its advantage in dealing with data imbalance problems.

[0189] (8) Diffusion model image generation experiment

[0190] We used seven malware families from the BIG 2015 dataset: Gatak, Kelihos_ver1, Obfuscator.ACY, Ramnit, Simda, Tracur, and Vundo. We used the diffusion model (DDIM) as the generative model and compared the quality of generated images using the Generative Adversarial Network (DCGAN) and the diffusion model (DDPM).

[0191] Table 9 Comparative experiments on generative models

[0192]

[0193] From the experimental results in Table 9, it can be seen that the diffusion model DDIM effectively overcomes the training instability and mode collapse problems of the generative adversarial network, especially achieving the lowest FID scores at both 128×128 and 224×224 image sizes, indicating that its generated images have the best quality.

[0194] (9) Zero-day malicious code detection experiment

[0195] We treated samples generated by DDIM as unknown malware samples and used them to simulate zero-day malware. We combined these generated samples with the original samples and trained the improved TransXNet model to evaluate the model's ability to detect zero-day malware. The experimental results are shown in Table 10.

[0196] Table 10 Zero-day malicious code detection experiment

[0197]

[0198] Experimental results show that when samples generated by DDIM are introduced, the accuracy of the model is improved to 99.54%, demonstrating the advantage of DDIM in generating high-quality samples. Further application of data augmentation technology (used to expand the training set by generating variant data, improve the generalization ability of the model, and prevent overfitting. Especially when the number of samples is limited, data augmentation technology can simulate the diversity of data and make the model more robust to different scenarios), the performance of the model reached an accuracy of 99.97%, reflecting the significant effect of data augmentation in improving the robustness and accuracy of the model. These results demonstrate the significant potential of the present invention in improving the accuracy and robustness of zero-day malicious code detection, and provide a more efficient and accurate solution for zero-day malicious code detection in complex environments.

[0199] from Figure 8 The confusion matrix shows that the model of this application has significantly improved the detection accuracy of all categories, further verifying the superiority of TransDiffusionNet in the zero-day malicious code detection task; the model effectively alleviates the confusion problem between categories, especially in accurately distinguishing difficult-to-identify malicious code samples; at the same time, through more precise feature extraction and fusion strategies, TransDiffusionNet demonstrates strong generalization and robustness, not only improving the recognition accuracy of complex malicious code samples, but also enhancing the ability to distinguish minority class samples. This result fully demonstrates the reliability and stability of the model constructed by this application in dealing with practical application scenarios.

[0200] (10) Ablation experiment

[0201] To better evaluate the improved TransDiffusionNet model, we gradually removed or modified certain key components and design choices in the TransDiffusionNet model to assess their impact on model performance. Therefore, in the ablation experiment, we controlled the TransXNet backbone network, visualization method, normalization method, EMA attention mechanism, Focal Loss loss function, and DDIM diffusion operation. The results of the TransDiffusionNet ablation experiment are shown in Table 11.

[0202] Table 11 Ablation experiment table

[0203]

[0204] The ablation experiments in Table 11 show that improving the malicious code visualization method alone increased model accuracy by 0.74 percentage points, demonstrating the positive impact of information integrity and feature diversity on model performance. Improving the normalization method increased accuracy from 98.06% to 98.39%, addressing the issue of texture feature changes during normalization, avoiding model misjudgments, and further improving detection accuracy.

[0205] After introducing the EMA module, accuracy increased by 0.46 percentage points, demonstrating that EMA enhances the model's multi-scale feature integration and contextual awareness capabilities. Although the overall improvement was small, ablation experiments using the Focal Loss function did improve model accuracy, validating its effectiveness in improving model performance.

[0206] Ablation experiments using DDIM demonstrate its importance in TransDiffusionNet. The introduction of DDIM improves accuracy by 0.91 percentage points. The generated samples mimic zero-day malware, and when combined with the original samples, significantly improve the model's detection accuracy and robustness.

[0207] Ablation experiments also verified the effectiveness of each module. These improvements not only ensure information integrity and feature diversity, avoid changes in texture features during normalization, but also enhance the ability to integrate multi-scale features. Furthermore, by generating high-quality samples, the efficiency and accuracy of zero-day malware detection are further improved.

Claims

1. A method for constructing a zero-day malicious code detection model, characterized in that: Taking TransXNet as the basic model, training and optimizing it to obtain the zero-day malicious code detection model, the method specifically includes: Preprocessing the malicious code sample data to obtain training set data; and Using the training set data to train and optimize TransXNet, the detection model is obtained; The preprocessing of the malicious code sample data includes: using a diffusion model DDIM to perform a diffusion operation on the visualized malicious code image to generate a new malicious code image, wherein the new malicious code image and the original malicious code image together constitute the training set data; The TransXNet adopts an improved TransXNet model, which includes: Multi-scale Attention Module (EMA), which extracts features from the input sequence through multi-scale convolution and pyramid pooling, and fuses features through a cross-spatial attention mechanism; Dynamic position encoding module DPE, used to dynamically generate position information of the input sequence; A lightweight dual dynamic token mixer (D-Mixer) for extracting global and local dynamic information from the fused features; and The multi-scale feedforward network MS-FFN is used to perform further cross-channel and multi-scale feature aggregation and pass the results to the output layer.

2. The method for constructing a zero-day malicious code detection model according to claim 1, wherein: The diffusion model DDIM is used to perform a diffusion operation on the visualized malicious code image to generate a new malicious code image. Specifically, the operation includes: adding noise to the data sample to gradually transform it into pure noise, and then gradually restoring the details and structure of the image through the trained inverse denoising process, thereby generating a new complex zero-day malicious code image.

3. The method for constructing a zero-day malicious code detection model according to claim 1, wherein: The pre-processing of the malicious code sample data further includes: Performing visualization processing on the malicious code to obtain a visualization image of the malicious code; And / or further perform image size normalization processing on the malicious code visualization image.

4. The method for constructing a zero-day malicious code detection model according to claim 3, wherein: The image size normalization process is performed using a bicubic interpolation algorithm and a sharpening technique, and / or the malicious code is visualized, including: Convert the binary file of malicious code into grayscale image; Convert assembly code files to RGB images; By fusing the grayscale image with the RGB image, we can obtain a visualization image of the malicious code.

5. The method for constructing a zero-day malicious code detection model according to any one of claims 1 to 4, characterized in that: The calculation formula of the loss function Focal Loss of the TransXNet model is: FL(p t )=-a t (1-p t ) γ log(p t ) Among them, p t is the model’s predicted probability for the true category, α t is a balancing parameter used to balance the ratio of positive and negative samples, and γ is an adjustment parameter used to adjust the weight of difficult and easy samples.

6. The method for constructing a zero-day malicious code detection model according to claim 5, wherein: The multi-scale attention module EMA consists of two parallel sub-networks, which extract and fuse features of the input sequence, specifically including: Divide the input feature map X into multiple sub-feature groups; The two sub-networks respectively process the sub-feature groups using different convolution kernels to obtain output feature maps of the two sub-networks; The output feature maps of the two sub-networks are then fused through the cross-spatial attention mechanism.

7. A zero-day malicious code detection model construction device, characterized in that: include: A training set data preprocessing unit, which is used to preprocess the malicious code sample data to obtain training set data; as well as A model training optimization unit, which is used to train and optimize TransXNet using the preprocessed training set data to obtain the detection model; The training set data preprocessing unit includes a diffusion module, which is used to perform a diffusion operation on the visualized malicious code image using a diffusion model DDIM to generate a new malicious code image, and the new malicious code image and the original malicious code image together constitute the training set data; The TransXNet adopts an improved TransXNet model, which includes: Multi-scale Attention Module (EMA), which extracts features from the input sequence through multi-scale convolution and pyramid pooling, and fuses features through a cross-spatial attention mechanism; Dynamic position encoding module DPE, used to dynamically generate position information of the input sequence; A lightweight dual dynamic token mixer (D-Mixer) for extracting global and local dynamic information from the fused features; and The multi-scale feedforward network MS-FFN is used to perform further cross-channel and multi-scale feature aggregation and pass the results to the output layer.

8. A zero-day malicious code detection method, characterized in that: include: Preprocess the zero-day malicious code to be detected to obtain a visual malicious code image; The visualized malicious code image is input into a zero-day malicious code detection model constructed by using any one of the methods in claims 1-6 to classify and detect the malicious code image and obtain a zero-day malicious code detection result.

9. The zero-day malicious code detection method according to claim 8, characterized in that: The pre-processing of the zero-day malicious code to be detected includes malicious code visualization processing and / or image size normalization processing, wherein: The visualization processing specifically includes: Convert the binary file of malicious code into grayscale image; Convert assembly code files to RGB images; The grayscale image is fused with the RGB image to obtain a malicious code visualization image; the image size normalization process is performed using a bicubic interpolation algorithm and a sharpening technique.

Images