Transform-based malicious code feature extraction and rule generation method and system

Abstract

The invention discloses a malicious code feature extraction and rule generation method based on Transform. The method comprises the steps that a malicious code sample is acquired and preprocessed; constructing a pre-training model, and training a classifier; model interpretation and key feature recognition and extraction; expanding a malicious family variant mode; generating a YARA rule; integrating with an anti-virus engine through a standardized interface; and obtaining expert experience adjustment data and applying the data. The invention further discloses a malicious code feature extraction and rule generation system based on the Transform. Full automation of feature extraction is realized through a Transform neural network, and large-scale malicious samples can be processed without manual intervention; multiple model interpretation technologies are integrated, key feature fragments can be accurately positioned, and a clear basis is provided for a detection result; the extracted features can be automatically converted into YARA rules which are directly used for an existing detection engine; a standardized interface is designed and can be seamlessly integrated with a mainstream anti-virus engine; and expert knowledge and data-driven feature learning are effectively combined.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a method and system for malicious code feature extraction and rule generation based on Transformer. Background Technology

[0002] Traditional signature-based detection methods rely mainly on security analysts manually extracting malicious code features and writing detection rules. While these methods have high accuracy, they suffer from low efficiency, high cost, and difficulty in dealing with the rapid growth of large-scale malicious samples.

[0003] To address the technical problems of traditional signature-based detection methods, those skilled in the art have proposed deep learning-based detection methods. These methods can improve the automation level of detection to a certain extent and achieve high accuracy in malware classification tasks. However, these methods primarily focus on improving classification performance, neglecting the practical application needs of feature extraction and rule generation, specifically exhibiting the following shortcomings:

[0004] 1. Insufficient precision and interpretability of feature extraction: Existing methods typically extract high-dimensional abstract features, which are difficult to directly map to specific code snippets or behavioral patterns. This results in insufficient interpretability of existing deep learning methods and an inability to provide clear evidence for detection results.

[0005] 2. Inability to generate usable detection rules: Currently, most deep learning methods cannot convert the learned features into a rule format that can be directly used by traditional detection engines, thus limiting their application in practical security products;

[0006] 3. Difficulty in integrating with existing engines: New technologies often require a complete replacement of existing detection engines, rather than working in conjunction with existing systems as enhancement modules;

[0007] 4. Insufficient utilization of expert experience: Existing methods cannot effectively utilize the domain knowledge and experience of security experts. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a method and system for malicious code feature extraction and rule generation based on Transformer.

[0009] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or to describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0010] The present invention adopts the following technical solution:

[0011] In a first aspect, the present invention provides a method for malicious code feature extraction and rule generation based on Transformer, comprising the following steps:

[0012] Obtain malicious code samples and convert them into a sequence format that can be processed by the Transformer model;

[0013] Build a pre-trained model and train the classifier;

[0014] By integrating the results of attention weight analysis, SHAP value calculation and gradient analysis, a comprehensive importance score for each input sequence is generated as the model interpretation result. Based on the model interpretation result, key feature fragments are extracted from the original malicious code sample.

[0015] The extracted key feature fragments are converted into structured YARA rules and mapped to their positions in the original binary file;

[0016] It integrates with antivirus engines through standardized interfaces.

[0017] Furthermore, the process of constructing the pre-trained model includes:

[0018] The training paradigm of masked language models is used to randomly mask parts of the format-converted sequence.

[0019] The Transformer encoder architecture is adopted, and self-supervised pre-training is performed on an unlabeled malicious code dataset to train the model to predict the masked content. The AdamW optimizer is used during training, and a warmup strategy is adopted for the learning rate.

[0020] The pre-training performance of the model was evaluated on the validation set, with metrics including mask prediction accuracy and perplexity.

[0021] The Transformer encoder architecture comprises 12 encoder layers, each containing 8 attention heads and 1 feedforward neural network. The model's hidden dimension is set to 768, supporting sequence inputs of length 2048. The vocabulary size is determined to be 256 based on byte-level encoding, and the positional encoding adopts learnable absolute positional encoding.

[0022] Furthermore, the process of training the classifier includes: performing supervised fine-tuning using labeled sample data based on the constructed pre-trained model;

[0023] In this process, a classifier structure consisting of a fully connected layer and a Softmax activation function is added to the output of the Transformer encoder to map the hidden state to the classification probability.

[0024] During training, the cross-entropy loss function is used as a supervision signal and the AdamW optimizer is used for parameter updates. Data augmentation is performed during training, and contrastive learning is used to enhance the model's ability to distinguish malicious families.

[0025] The data augmentation methods include: random byte replacement, sequence truncation, and noise injection;

[0026] The contrastive learning method includes: bringing the representations of similar samples closer together and pushing the representations of dissimilar samples further apart.

[0027] Furthermore, the process of generating a comprehensive importance score for each input sequence by fusing the results of attention weight analysis, SHAP value calculation, and gradient analysis includes:

[0028] Extract the self-attention weight matrix of each layer of the Transformer encoder, and calculate the importance score of each input sequence based on the attention distribution;

[0029] The DeepSHAP algorithm is used to calculate the SHAP value of each input sequence, and the SHAP value is used as the contribution value to the final prediction result.

[0030] Calculate the absolute value of the gradient of the loss function with respect to each input sequence;

[0031] The three calculated scores are weighted and fused to obtain the final comprehensive importance score of the input sequence, where the weight corresponding to the attention weight analysis result is 0.4, the weight corresponding to the SHAP value calculation result is 0.4, and the weight corresponding to the gradient analysis result is 0.2.

[0032] Furthermore, the process of extracting key feature fragments from the original malicious code sample based on the model interpretation results includes:

[0033] A sliding window method is used to traverse malicious code samples, and the boundaries of feature segments are determined based on the comprehensive importance score. The sliding window method uses a threshold method to determine the segment boundaries.

[0034] The extracted feature fragments are subjected to semantic integrity verification, including determining whether the API call sequence is complete, whether the byte sequence conforms to the predefined structure, and whether the string constant has a valid format.

[0035] The feature fragments that pass semantic verification are filtered, wherein the filtering operation includes: removing duplicate feature fragments and retaining feature fragments that have stable discrimination ability in multiple types of malicious code samples;

[0036] The feature fragments include at least one of the following types: API call pattern, string constant, byte sequence signature, and control flow feature.

[0037] Furthermore, the process of converting the extracted key feature fragments into structured YARA rules and mapping them to their positions in the original binary file includes:

[0038] Byte sequence rule generation: Record the offset position of key feature fragments in the original malicious sample file, convert the feature fragments into hexadecimal mode and retain the file offset information, insert wildcards at the changing positions in the byte mode, and add file offset constraints to the rules;

[0039] String matching rule generation: Extract string constants from the original malicious sample file and record the file offset and encoding format of the string constants. Convert the strings into YARA rule strings and add position constraints to the rules.

[0040] API call sequence rule generation: Extract the API call sequence and its position in the import table, map the API calls to specific entries in the PE file import table, and generate a detection pattern based on the order of imported functions;

[0041] Condition combination rule generation: Generate spatial constraints based on the file positional relationship of feature fragments, add distance constraints between features, and add PE section constraints.

[0042] Furthermore, the process of integrating with the antivirus engine through a standardized interface includes:

[0043] The generated YARA rules are added to the virus database via ClamAV's signature database interface for integration with the ClamAV engine;

[0044] By calling the Windows Security API, the detection logic is integrated into the Windows Defender scanning engine to achieve integration with the Windows Defender engine;

[0045] Design a RESTful API interface that provides sample submission, result query, and rule update capabilities for integration with general interfaces.

[0046] Furthermore, the method for malicious code feature extraction and rule generation based on Transformer further includes: providing a graphical interface to obtain weight values ​​after security experts have adjusted the weights for different feature types; providing a sample annotation interface to obtain annotation results from security experts who have verified and corrected the features extracted by the model; establishing a security knowledge base to store the experience and threat intelligence of security experts; and adjusting model parameters and feature weights according to the weight values ​​after weight adjustment and the annotation results.

[0047] Furthermore, the malicious code samples include: PE files, ELF files, script files, and source code;

[0048] The conversion to a sequence format that the Transformer model can process includes: reading the binary file byte by byte and converting it into an integer sequence of 0-255; extracting the API call sequence of the program using a static analysis tool; and converting the extracted sequence into the token sequence format input to the Transformer model.

[0049] Secondly, a system for malicious code feature extraction and rule generation based on Transformer is also provided, including:

[0050] The data input layer is used to obtain malicious code samples;

[0051] The preprocessing layer is used to convert the acquired malicious code samples into a sequence format that can be processed by the Transformer model;

[0052] The Transformer core layer is used to build pre-trained models;

[0053] Classifier layer, used to train the classifier;

[0054] The model interpretation layer is used to generate a comprehensive importance score for each input sequence by fusing the results of attention weight analysis, SHAP value calculation and gradient analysis as the model interpretation result, and to extract key feature fragments from the original malicious code sample based on the model interpretation result.

[0055] The rule generation layer is used to convert the extracted key feature fragments into structured YARA rules and map them to their positions in the original binary file;

[0056] Engine integration layer, used for integration with antivirus engines through standardized interfaces.

[0057] The beneficial effects of this invention are as follows:

[0058] High degree of automation: Fully automated feature extraction is achieved through Transformer neural networks, which can process large-scale malicious samples without human intervention;

[0059] High interpretability: It integrates multiple model interpretation technologies, which can accurately locate key feature segments and provide clear evidence for detection results;

[0060] Rule generation capability: It can automatically convert extracted features into YARA rules, which can be directly used in existing detection engines;

[0061] Good engine compatibility: The standardized interface design allows for seamless integration with mainstream antivirus engines;

[0062] Expert experience integration: Provides a weight adjustment mechanism to effectively combine expert knowledge and data-driven feature learning. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating a method for malicious code feature extraction and rule generation based on Transformer according to the present invention. Detailed Implementation

[0065] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0066] like Figure 1 As shown, a method for malicious code feature extraction and rule generation based on Transformer is provided, including the following steps:

[0067] 101: Malicious code sample input.

[0068] The process involves acquiring malware samples, including PE files, ELF files, script files, and source code. To obtain diverse malware samples, the collected large-scale malware samples must cover both known malware families and unknown samples to ensure data diversity and representativeness, and support both single-sample analysis and batch sample processing, thus making it suitable for large-scale dataset training and detection tasks.

[0069] 102: Preprocess the obtained malicious code samples.

[0070] The acquired malicious code sample is converted into a sequence format that can be processed by the Transformer model, specifically the token sequence format input to the Transformer model, including the following steps:

[0071] Converting a binary file to a byte sequence:

[0072] The binary file is read byte by byte and converted into an integer sequence of 0-255. In specific implementation, for example, the .text and .data segments of the PE file are read and a byte sequence of length 2048 is extracted. For very long files, the sliding window technique is used for block processing to ensure that no information is lost. Each byte value (0-255) is mapped to a unique token ID to build a byte-level vocabulary.

[0073] API call sequence extraction:

[0074] The API call sequence of the program is extracted by static analysis tools, an API vocabulary is built, including common Windows API functions, etc., and the API name is mapped to a unique token ID to form an API call token sequence.

[0075] 103: Construct a pre-trained model. This includes the following steps:

[0076] First, using the training paradigm of the masked language model, parts of the format-converted sequence are randomly masked. The masking strategy is to randomly select 15% of the tokens in the input sequence for masking: 80% are replaced with the special tag [MASK], 10% are replaced with random tokens, and 10% remain unchanged.

[0077] Then, a Transformer encoder architecture is employed for self-supervised pre-training on an unlabeled malware dataset. The loss function of a masked language model is used to train the model to predict the masked content; that is, the goal is to predict the masked token. The model predicts the masked token through context, thereby learning the semantic relationship between byte sequences and API call sequences. This pre-training method enables the model to learn the inherent semantic patterns of malware, such as the meaning of specific byte combinations, and structural features, such as the order and dependencies of API calls.

[0078] The Transformer encoder architecture is specifically configured as follows: it contains 12 encoder layers, each layer contains 8 attention heads and 1 feedforward neural network, the hidden dimension of the model is set to 768, it supports sequence inputs of length 2048, the vocabulary size is determined to be 256 based on byte-level encoding, the position encoding adopts learnable absolute position encoding, and it supports a maximum sequence length of 2048 tokens.

[0079] The multi-head self-attention mechanism can focus on the relationships between different positions in the input sequence in parallel, enabling the model to capture long-distance dependencies between tokens to adapt to the complex structures in malicious code. Simultaneously, each attention layer is followed by a feedforward neural network for non-linear transformation and feature extraction. The model's hidden dimension represents the vector dimension of its internal representation, determining its expressive power; the vocabulary size is 256, corresponding to all possible byte values ​​from 0x00 to 0xFF; learnable absolute positional encoding is used to mark the position of tokens in the sequence, supporting a maximum sequence length of 2048 tokens, sufficient to handle relatively long malicious code sequences.

[0080] The training strategy is as follows: the AdamW optimizer is used during training, combined with weight decay to improve generalization ability; the learning rate adopts a warmup strategy, that is, the initial learning rate increases linearly to a set value, and then the cosine annealing strategy is used to gradually decrease it; cross-entropy loss is used to calculate the prediction error of the model on the mask token.

[0081] Finally, the pre-training performance of the model was evaluated on an independent validation set. The metrics included mask prediction accuracy and perplexity. Mask prediction accuracy refers to the proportion of masked tokens that the model correctly predicts. Perplexity measures the uncertainty of the model's prediction of the sequence. The lower the value, the better the model performance.

[0082] The training method of this application has strong self-supervised learning capabilities, requires no manual data labeling, and is suitable for domains that are difficult to label, such as malicious code. Moreover, it has rich semantic representation. The mask language model training method enables the model to learn the contextual dependencies and structural patterns of malicious code, which is suitable for long sequence modeling and complex feature extraction. At the same time, the combination of AdamW optimizer and warmup strategy can improve training stability and convergence speed, ensuring that the training strategy is stable and efficient. In addition, the dual evaluation of mask prediction accuracy and perplexity can achieve a comprehensive measurement of the model pre-training effect.

[0083] 104: Training the classifier. This specifically includes: supervised fine-tuning using labeled sample data based on the constructed pre-trained model.

[0084] During the pre-training phase, the model gains a general understanding of malicious code through self-supervised learning using a masked language model, but it is not optimized for specific tasks such as malicious code detection and classification. Therefore, it is necessary to use labeled data, such as known malicious / benign samples, to perform supervised fine-tuning of the model to adapt it to downstream tasks.

[0085] Specifically, a classifier structure consisting of fully connected layers and a Softmax activation function is added to the output of the Transformer encoder to map the hidden states to classification probabilities. The fully connected layers map the high-dimensional features output by the pre-trained Transformer model to the class space, such as for malicious / benign binary classification. The Softmax activation function transforms the output of the fully connected layers into a probability distribution, representing the predicted probability of each class. For binary classification tasks, the output dimension is 2; for multi-class classification tasks, the output dimension equals the number of classes.

[0086] Training strategy: A progressive fine-tuning strategy is adopted. First, the parameters of the Transformer encoder are frozen, and only the classification head is trained. Then, the layers of the encoder are gradually unfrozen to perform end-to-end fine-tuning.

[0087] During training, the cross-entropy loss function is used as a supervision signal, and the AdamW optimizer is employed for parameter updates. Data augmentation is also performed during training, and contrastive learning is used to enhance the model's ability to distinguish malicious families. Data augmentation methods include random byte replacement, sequence truncation, and noise injection; contrastive learning methods include bringing similar samples closer together and pushing dissimilar samples further apart.

[0088] The cross-entropy loss function is used to measure the difference between the model's predicted probability and the true label. The AdamW optimizer incorporates weight decay, which can effectively prevent overfitting and improve the model's generalization ability.

[0089] Meanwhile, to further improve the model's generalization ability, various data augmentation methods were used during training:

[0090] Random byte replacement: Randomly select some bytes in the input sequence and replace them with other random bytes to simulate mutations or obfuscations in malicious code, thereby enhancing the model's robustness to perturbations;

[0091] Sequence truncation: Randomly truncate part of the input sequence, retaining the head or tail, to simulate malicious code samples of different lengths, so that the model can adapt to different input lengths;

[0092] Noise injection: Adding a small amount of noise, such as random bytes or perturbations, to the input sequence enhances the model's robustness to noise and adversarial examples.

[0093] Multiple data augmentation methods enable this application to generate diverse training samples, allowing the model to maintain high detection and classification performance even when faced with ever-changing malicious code.

[0094] Contrastive learning primarily focuses on learning the similarity between samples. By bringing the representations of similar samples closer together and pushing the representations of dissimilar samples further apart, in this embodiment, it is specifically used to enhance the model's ability to distinguish between different malicious families. For samples from the same family, the model learns to bring their representation vectors closer together in the feature space, i.e., reduce the distance; for samples from different families, the model learns to push their representation vectors further apart, i.e., increase the distance. Implementing contrastive learning allows the model to not only focus on the features of individual samples but also learn the relationships between samples, thereby improving its ability to distinguish malicious families.

[0095] Through supervised fine-tuning and data augmentation, the model becomes more accurate in detecting and classifying malicious code. It can maintain stable performance even when faced with variations, obfuscation, and noise in malicious code. Contrastive learning enables the model to distinguish between different malicious families, making it suitable for more complex security analysis tasks.

[0096] 105: Model interpretation and identification and extraction of key features.

[0097] By integrating the results of attention weight analysis, SHAP value calculation, and gradient analysis, a comprehensive importance score for each input sequence is generated as the model interpretation result. Based on the model interpretation result, key feature fragments are extracted from the original malicious code sample.

[0098] Specifically, the process of generating a comprehensive importance score for each input sequence includes the following steps:

[0099] Extract the self-attention weight matrix of each layer of the Transformer encoder, and calculate the importance score of each input sequence based on the attention distribution. The attention weight matrix represents the degree of attention the model pays to other tokens when processing each token. Tokens with high attention weights usually correspond to important feature fragments, such as key instructions or sensitive API calls in malicious code. Therefore, the tokens that the model focuses on can be selected by analyzing the attention distribution.

[0100] In practice, the input sequence is first obtained, and the self-attention weight matrix of each layer in the Transformer model is extracted. Then, the attention weights of the multiple layers are fused into a single attention matrix, either by mean or weighted average. Finally, the attention rows of each token are summed or the maximum value is taken to obtain the importance score of each input sequence. The input positions can be ranked according to their importance scores, thereby identifying the token positions that the model focuses on.

[0101] The DeepSHAP algorithm is used to calculate the SHAP value for each input sequence, and this SHAP value is used as the contribution to the final prediction result. SHAP is a feature attribution method based on game theory. DeepSHAP is an extension of SHAP in deep learning and is applicable to neural network models. The sign of the SHAP value indicates the direction of the token's influence on the prediction result: a positive value indicates that the token increases the probability of malicious judgment; a negative value indicates that the token decreases the probability of malicious judgment.

[0102] In implementation, the token contribution value is first calculated based on the DeepSHAP algorithm. A suitable baseline input, such as the zero vector or the average vector, is selected. The SHAP values ​​of each layer are summarized to generate the final SHAP score for each token. The absolute value of the SHAP score reflects the importance of the token. The importance of the input positions can be ranked according to the absolute value of the SHAP score, thereby identifying the token positions that the model focuses on.

[0103] Calculate the absolute value of the gradient of the loss function with respect to each input sequence. Gradient analysis measures the sensitivity of a token to the model's decision by calculating the gradient of the loss function with respect to the input token. A larger gradient indicates that a small change in the token has a greater impact on the model's output. The importance is usually measured using the absolute value of the gradient or the squared gradient.

[0104] In this embodiment, the gradient of the loss function with respect to the input token is first calculated, and the absolute value of the gradient is taken. Based on the magnitude of the absolute value of the gradient, the input position that is sensitive to the model's decision is identified, and the gradient score is standardized to facilitate comparison with other scores.

[0105] The three calculated scores are weighted and fused to obtain the final comprehensive importance score of the input sequence, where the weight corresponding to the attention weight analysis result is 0.4, the weight corresponding to the SHAP value calculation result is 0.4, and the weight corresponding to the gradient analysis result is 0.2.

[0106] Attention weight analysis, SHAP value calculation, and gradient analysis construct a multi-dimensional feature importance evaluation method. This multi-faceted approach not only reveals the key areas the model focuses on during decision-making but also comprehensively evaluates the input sequence from different perspectives (attention distribution, feature contribution, and sensitivity), significantly improving the interpretability and credibility of the model's decisions. By outputting the importance score for each token, analysts can intuitively understand which segments are the key evidence for the model's "malicious" judgment, providing a clear basis for manual review, rule extraction, and model tuning. Furthermore, by weightedly fusing attention weights, SHAP values, and gradient information, the contributions of different interpretive perspectives are balanced, which helps to accurately locate key feature segments and automatically extract the most discriminative feature segments from the original malicious code samples, such as sensitive API calls, key instructions, and abnormal byte sequences.

[0107] Finally, based on the model interpretation results, key feature fragments are extracted from the original malicious code sample, specifically including the following steps:

[0108] A sliding window method is used to traverse malicious code samples, and the boundaries of feature segments are determined based on the comprehensive importance score. The sliding window method employs a threshold method to determine segment boundaries, with the threshold set at the 90th percentile of the fusion score. Further rules are defined: consecutive high-importance tokens are merged into a single feature segment, and isolated high-importance tokens are expanded outwards into their respective contexts until semantic completeness is achieved.

[0109] The extracted feature fragments are subjected to semantic integrity verification, including determining whether the API call sequence is complete (e.g., parameter matching, call order), whether the byte sequence conforms to the predefined structure (e.g., conforms to PE structure, encryption algorithm features), and whether the string constant has a valid format.

[0110] The feature fragments that pass semantic verification are filtered. The filtering operation includes: removing duplicate feature fragments and retaining feature fragments that have stable discrimination ability in multiple types of malicious code samples.

[0111] The feature fragments include at least one of the following types: API call patterns, string constants, byte sequence signatures, and control flow features. API call patterns are continuous sequences of API calls, especially combinations involving file, network, or system modifications; string constants are hard-coded strings, such as URLs, paths, and registry keys; byte sequence signatures are discriminative byte patterns, such as encrypted constants and PE header identifiers; control flow features include abnormal jumps and loop structures, which may indicate anti-debugging or obfuscation techniques.

[0112] 106: Malware Family Variant Pattern Expansion. Analysis of extracted feature fragments reveals other variant samples within the same malware family. Specific methods include:

[0113] Fuzzy hash matching: Using fuzzy hash algorithms, such as ssdeep, the similarity between samples is calculated, that is, the fuzzy hash value between samples, in order to identify variants with similar structures. Specifically, it can be based on the similarity threshold of the hash values ​​to determine whether the samples belong to the same malicious family.

[0114] Feature vector clustering: Convert the feature fragments of a sample into a vector representation, use clustering algorithms such as K-means and DBSCAN to cluster the samples, divide the samples into different family categories based on the clustering results, and identify potential variants;

[0115] Graph matching algorithm: The relationship between feature fragments is modeled as a graph structure, that is, the feature fragments in the sample are represented as nodes of the graph, and the dependency or co-occurrence relationship between feature fragments is represented as edges. Then, the graph isomorphism algorithm is used to identify malware variants with similar patterns. Specifically, the graph structures of different samples can be compared. If the graph structures of two samples meet the isomorphism condition, they are determined to be variants belonging to the same malware family.

[0116] By utilizing extracted feature fragments and combining fuzzy hashing, feature clustering, and graph matching, it can effectively identify deeply obfuscated and variant samples that are difficult to detect using traditional methods, and provide highly readable evidence, such as which functions are similar and how the calling logic matches, significantly enhancing the ability to identify malware variants, analysis efficiency, and interpretability of results.

[0117] 107: Generate YARA rules. The extracted key feature fragments are converted into structured YARA rules and mapped to their positions in the original binary file. This includes: byte sequence rule generation, string matching rule generation, API call sequence rule generation, and condition combination rule generation.

[0118] Byte sequence rule generation: Record the offset position of key feature fragments in the original malicious sample file, convert the feature fragments to hexadecimal mode and retain the file offset information, insert wildcards at the changing positions in the byte mode, and add file offset constraints to the rules.

[0119] In practice, the first step is to locate the offset (start and end addresses) of the key feature fragment in the original PE file. This feature fragment is then converted into a hexadecimal string format supported by YARA, while preserving the file offset information. Simultaneously, to address minor differences that may occur in malware variants, wildcards (??) or jumps ([xy]) are inserted at potentially changing positions in the hexadecimal pattern (such as offsets or padding bytes). For example, a pattern like { 4D 5A ?? ?? [0-100]50 45} is generated. Finally, to improve rule targeting and reduce false positives, a file offset constraint is added to this hexadecimal pattern, such as $hex_pattern at 0x1000-0x2000, limiting its matching range to a specific region within the file.

[0120] String matching rule generation: Extract string constants from the original malicious sample file and record the file offset and encoding format of the string constants. Convert the strings into YARA rule strings and add position constraints to the rules.

[0121] In practice, hard-coded strings are extracted from binary files, and the encoding format of the strings is identified. Common encoding formats include ASCII, Unicode, and UTF-8. After the encoding is identified, the exact position information of the string in the file is recorded to ensure the accuracy of the test. Position constraints are added to the rules to reduce false alarms.

[0122] API call sequence rule generation: Extract the API call sequence and its position in the import table, map the API calls to specific entries in the PE file import table, and generate a detection pattern based on the order of imported functions.

[0123] In practice, the PE file structure is parsed to locate the import table. Each imported descriptor is traversed to obtain the name of the imported DLL and the function address table pointed to by OriginalFirstThunk or FirstThunk, thereby extracting all API function names. The extracted list of API functions is then converted into detection rules, including: recording all extracted API names, applicable to malicious families with obvious characteristics, and recording the order in which API functions appear in the import table. When generating rules, the approximate offset or index range of key APIs in the import table of their respective DLLs can be recorded as enhanced detection conditions.

[0124] Condition combination rule generation: Generate spatial constraints based on the file positional relationship of feature fragments, add distance constraints between features, and add PE section constraints.

[0125] Spatial constraints:

[0126] Spatial constraints primarily utilize the layout relationships of features within a file to generate rules. Relative position constraints specify the order in which features appear; for example, feature A must appear before feature B. Absolute offset constraints use operators to confine features to specific regions within the file.

[0127] Distance constraints:

[0128] By constraining the byte distance (offset difference) between two or more features, a stable internal layout in malicious code can be captured, effectively countering simple obfuscation such as overall code translation. Precise distance: requires a fixed number of bytes between two features. Range distance: constrains the distance between two features to a minimum and maximum value, providing some flexibility.

[0129] PE section constraints:

[0130] Limit the search scope of features to specific PE sections. Many malicious codes store their core functions or data in specific sections (such as .text, .data, .rdata), or create sections with unconventional names. Utilizing this can significantly reduce false positives.

[0131] 108: Integrate with antivirus engines through standardized interfaces. This includes the following steps:

[0132] The generated YARA rules are added to the virus database via ClamAV's signature database interface for integration with the ClamAV engine;

[0133] By calling the Windows Security API, the detection logic is integrated into the Windows Defender scanning engine to achieve integration with the Windows Defender engine;

[0134] Design a RESTful API interface that provides sample submission, result query, and rule update capabilities for integration with general interfaces.

[0135] Through standardized interface design, seamless integration with mainstream antivirus engines such as ClamAV and Windows Defender is achieved. This cross-platform compatibility allows the generated YARA rules to be directly applied to malware detection in different environments, significantly improving the applicability and practical value of the rules. The designed RESTful API uses JSON format for data exchange, supports asynchronous processing and status queries, and provides a flexible integration method for third-party security platforms.

[0136] 109: Acquire expert experience to adjust data and apply it to adjust model parameters and feature weights, specifically including:

[0137] Provides a graphical interface to obtain the weight values ​​after security experts have adjusted the weights for different feature types, so that experts can increase the importance of certain features based on experience;

[0138] Provides a sample annotation interface to obtain annotation results from security experts who have verified and corrected the features extracted by the model. The annotation results are used for continuous model optimization.

[0139] Establish a security knowledge base to store the experience and threat intelligence of security experts. The knowledge base is linked with the feature extraction module to guide the identification and verification of features.

[0140] The model parameters and feature weights are adjusted based on the weight values ​​after weight adjustment and the annotation results to achieve continuous learning and optimization of the system.

[0141] This application presents a method for extracting malicious code features and generating rules based on Transformer, which can be applied to malicious code detection, static analysis engines, antivirus software, and network security protection systems. It automatically extracts key feature fragments of malicious code through deep learning technology and generates rules and signatures that can be used in existing detection engines.

[0142] Compared to existing technologies, it has the following significant advantages:

[0143] 1. Improve processing efficiency.

[0144] By utilizing the Transformer neural network, the entire process from inputting malicious code samples to feature extraction and rule generation is automated. This allows for efficient processing of large-scale malicious samples without human intervention, significantly reducing the cost of manual analysis and improving detection efficiency.

[0145] 2. Strong interpretability, accurately locating key features.

[0146] By integrating multiple model interpretation technologies, this invention can accurately locate key feature fragments in malicious code and clarify their position in the original binary file, which not only improves the interpretability of the model, but also provides security analysts with clear and traceable detection basis.

[0147] 3. Achieve integration with existing detection engines.

[0148] The extracted key features are transformed into structured YARA rules, supporting multiple rule types such as byte sequences, strings, and API call sequences. These rules can be directly imported into mainstream antivirus engines such as ClamAV and Windows Defender, greatly enhancing the practicality and adaptability of the detection system. The standardized interface design ensures compatibility with mainstream antivirus engines.

[0149] 4. Incorporation of expert experience.

[0150] It provides a weight adjustment mechanism, allowing security experts to intervene and optimize the feature extraction process. Combining expert experience with the model not only improves the accuracy and robustness of feature extraction, but also enhances the system's adaptability to new threats.

[0151] A system for malicious code feature extraction and rule generation based on Transformer is provided, including:

[0152] The data input layer is used to obtain malicious code samples.

[0153] The preprocessing layer is used to convert the acquired malicious code samples into a sequence format that can be processed by the Transformer model.

[0154] The Transformer core layer is used to build pre-trained models.

[0155] The classifier layer is used to train the classifier.

[0156] The model interpretation layer is used to generate a comprehensive importance score for each input sequence by fusing the results of attention weight analysis, SHAP value calculation and gradient analysis as the model interpretation result, and extract key feature fragments from the original malicious code sample based on the model interpretation result.

[0157] The rule generation layer is used to convert the extracted key feature fragments into structured YARA rules and map them to their positions in the original binary file.

[0158] Engine integration layer, used for integration with antivirus engines through standardized interfaces.

[0159] The expert system layer is used to acquire and apply expert experience to adjust data, thereby adjusting the model's parameters and feature weights.

[0160] The preprocessing layer includes: a first conversion module for reading the binary file byte by byte and converting it into an integer sequence of 0-255; a second conversion module for extracting the API call sequence of the program using static analysis tools; and converting the extracted sequence into the token sequence format input to the Transformer model.

[0161] The Transformer core layer uses a masked language model training paradigm to randomly mask parts of the format-transformed sequence; it employs a Transformer encoder architecture and performs self-supervised pre-training on an unlabeled malware dataset to train the model to predict the masked content. During training, the AdamW optimizer is used, and a warmup strategy is adopted for the learning rate; the pre-training performance of the model is evaluated on a validation set, with metrics including mask prediction accuracy and perplexity.

[0162] The Transformer encoder architecture consists of 12 encoder layers, each containing 8 attention heads and 1 feedforward neural network. The model's hidden dimension is set to 768, supporting sequence inputs of length 2048. The vocabulary size is determined to be 256 based on byte-level encoding, and the positional encoding adopts learnable absolute positional encoding.

[0163] The classifier layer is fine-tuned in a supervised manner using labeled sample data based on the pre-trained model. Specifically, a classifier structure consisting of a fully connected layer and a Softmax activation function is added to the output of the Transformer encoder to map the hidden states to classification probabilities.

[0164] During training, the cross-entropy loss function is used as a supervision signal and the AdamW optimizer is used for parameter updates. Data augmentation is also performed during training, and contrastive learning is used to enhance the model's ability to distinguish malicious families.

[0165] Data augmentation methods include: random byte replacement, sequence truncation, and noise injection.

[0166] Contrastive learning methods include: bringing the representations of similar samples closer together and pushing the representations of dissimilar samples further apart.

[0167] The model interpretation layer includes:

[0168] The first calculation module is used to extract the self-attention weight matrix of each layer of the Transformer encoder and calculate the importance score of each input sequence based on the attention distribution.

[0169] The second calculation module is used to calculate the SHAP value of each input sequence using the DeepSHAP algorithm, and uses the SHAP value as the contribution value to the final prediction result.

[0170] The third calculation module is used to calculate the absolute value of the gradient of the loss function with respect to each input sequence.

[0171] The fusion output module is used to weight and fuse the three calculated scores to obtain the final comprehensive importance score of the input sequence. The weight corresponding to the attention weight analysis result is 0.4, the weight corresponding to the SHAP value calculation result is 0.4, and the weight corresponding to the gradient analysis result is 0.2.

[0172] The identification and extraction module is used to traverse malicious code samples using a sliding window method, determine the boundaries of feature segments based on a comprehensive importance score, wherein the sliding window method uses a threshold method to determine the segment boundaries; perform semantic integrity verification on the extracted feature segments, including determining whether the API call sequence is complete, whether the byte sequence conforms to a predefined structure, and whether the string constant has a valid format; and perform filtering operations on the feature segments that pass the semantic verification, wherein the filtering operations include: removing duplicate feature segments and retaining feature segments that have stable discrimination ability among multiple types of malicious code samples;

[0173] The feature fragments include at least one of the following types: API call pattern, string constant, byte sequence signature, and control flow feature.

[0174] The rule generation layer includes:

[0175] The first generation module is used for byte sequence rule generation. It records the offset position of key feature fragments in the original malicious sample file, converts the feature fragments into hexadecimal mode and retains the file offset information, inserts wildcards at the changing positions in the byte mode, and adds file offset constraints to the rules.

[0176] The second generation module is used to generate string matching rules. It extracts string constants from the original malicious sample file and records the file offset and encoding format of the string constants. It converts the strings into YARA rule strings and adds position constraints to the rules.

[0177] The third generation module is used to generate API call sequence rules. It extracts the API call sequence and its position in the import table, maps the API calls to specific entries in the PE file import table, and generates a detection pattern based on the order of imported functions.

[0178] The fourth generation module is used to generate condition combination rules. It generates spatial constraints based on the file positional relationship of feature fragments, adds distance constraints between features, and adds PE section constraints.

[0179] The engine integration layer adds the generated YARA rules to the virus database through ClamAV's signature database interface to integrate with the ClamAV engine; it integrates the detection logic into Windows Defender's scanning engine by calling the Windows Security API to integrate with the Windows Defender engine; and it designs a RESTful API interface to provide sample submission, result query, and rule update functions to integrate with the general interface.

[0180] The expert system layer provides a graphical interface to obtain the weight values ​​after security experts have adjusted the weights for different feature types; it provides a sample annotation interface to obtain the annotation results of security experts in verifying and correcting the features extracted by the model; it establishes a security knowledge base to store the experience and threat intelligence of security experts; and it adjusts the model parameters and feature weights according to the weight values ​​after weight adjustment and the annotation results.

[0181] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for malicious code feature extraction and rule generation based on Transformer, characterized in that, Includes the following steps: Obtain malicious code samples and convert them into a sequence format that can be processed by the Transformer model; Build a pre-trained model and train the classifier; By integrating the results of attention weight analysis, SHAP value calculation and gradient analysis, a comprehensive importance score for each input sequence is generated as the model interpretation result. Based on the model interpretation result, key feature fragments are extracted from the original malicious code sample. The extracted key feature fragments are converted into structured YARA rules and mapped to their positions in the original binary file; It integrates with antivirus engines through standardized interfaces.

2. The method for malicious code feature extraction and rule generation based on Transformer according to claim 1, characterized in that, The process of building the pre-trained model includes: The training paradigm of masked language models is used to randomly mask parts of the format-converted sequence. The Transformer encoder architecture is adopted, and self-supervised pre-training is performed on an unlabeled malicious code dataset to train the model to predict the masked content. The AdamW optimizer is used during training, and a warmup strategy is adopted for the learning rate. The pre-training performance of the model was evaluated on the validation set, with metrics including mask prediction accuracy and perplexity. The Transformer encoder architecture comprises 12 encoder layers, each containing 8 attention heads and 1 feedforward neural network. The model's hidden dimension is set to 768, supporting sequence inputs of length 2048. The vocabulary size is determined to be 256 based on byte-level encoding, and the positional encoding adopts learnable absolute positional encoding.

3. The method for malicious code feature extraction and rule generation based on Transformer according to claim 2, characterized in that, The process of training the classifier includes: performing supervised fine-tuning using labeled sample data based on the constructed pre-trained model; In this process, a classifier structure consisting of a fully connected layer and a Softmax activation function is added to the output of the Transformer encoder to map the hidden state to the classification probability. During training, the cross-entropy loss function is used as a supervision signal and the AdamW optimizer is used for parameter updates. Data augmentation is performed during training, and contrastive learning is used to enhance the model's ability to distinguish malicious families. The data augmentation methods include: random byte replacement, sequence truncation, and noise injection; The contrastive learning method includes: bringing the representations of similar samples closer together and pushing the representations of dissimilar samples further apart.

4. The method for malicious code feature extraction and rule generation based on Transformer according to claim 3, characterized in that, The process of generating a comprehensive importance score for each input sequence by fusing the results of attention weight analysis, SHAP value calculation, and gradient analysis includes: Extract the self-attention weight matrix of each layer of the Transformer encoder, and calculate the importance score of each input sequence based on the attention distribution; The DeepSHAP algorithm is used to calculate the SHAP value of each input sequence, and the SHAP value is used as the contribution value to the final prediction result. Calculate the absolute value of the gradient of the loss function with respect to each input sequence; The three calculated scores are weighted and fused to obtain the final comprehensive importance score of the input sequence, where the weight corresponding to the attention weight analysis result is 0.4, the weight corresponding to the SHAP value calculation result is 0.4, and the weight corresponding to the gradient analysis result is 0.

2.

5. The method for malicious code feature extraction and rule generation based on Transformer according to claim 4, characterized in that, The process of extracting key feature fragments from the original malicious code sample based on the model interpretation results includes: A sliding window method is used to traverse malicious code samples, and the boundaries of feature segments are determined based on the comprehensive importance score. The sliding window method uses a threshold method to determine the segment boundaries. The extracted feature fragments are subjected to semantic integrity verification, including determining whether the API call sequence is complete, whether the byte sequence conforms to the predefined structure, and whether the string constant has a valid format. The feature fragments that pass semantic verification are filtered, wherein the filtering operation includes: removing duplicate feature fragments and retaining feature fragments that have stable discrimination ability in multiple types of malicious code samples; The feature fragments include at least one of the following types: API call pattern, string constant, byte sequence signature, and control flow feature.

6. The method for malicious code feature extraction and rule generation based on Transformer according to claim 5, characterized in that, The process of converting the extracted key feature fragments into structured YARA rules and mapping them to their positions in the original binary file includes: Byte sequence rule generation: Record the offset position of key feature fragments in the original malicious sample file, convert the feature fragments into hexadecimal mode and retain the file offset information, insert wildcards at the changing positions in the byte mode, and add file offset constraints to the rules; String matching rule generation: Extract string constants from the original malicious sample file and record the file offset and encoding format of the string constants. Convert the strings into YARA rule strings and add position constraints to the rules. API call sequence rule generation: Extract the API call sequence and its position in the import table, map the API calls to specific entries in the PE file import table, and generate a detection pattern based on the order of imported functions; Condition combination rule generation: Generate spatial constraints based on the file positional relationship of feature fragments, add distance constraints between features, and add PE section constraints.

7. The method for malicious code feature extraction and rule generation based on Transformer according to claim 6, characterized in that, The process of integrating with the antivirus engine through a standardized interface includes: The generated YARA rules are added to the virus database via ClamAV's signature database interface for integration with the ClamAV engine; By calling the Windows Security API, the detection logic is integrated into the Windows Defender scanning engine to achieve integration with the Windows Defender engine; Design a RESTful API interface that provides sample submission, result query, and rule update capabilities for integration with general interfaces.

8. The method for malicious code feature extraction and rule generation based on Transformer according to claim 7, characterized in that, Also includes: Provides a graphical interface to obtain the weight values ​​after security experts have adjusted the weights for different feature types; Provide a sample annotation interface to obtain annotation results from security experts who verify and correct the features extracted by the model; establish a security knowledge base to store the experience and threat intelligence of security experts; and adjust the model parameters and feature weights based on the weight values ​​after weight adjustment and the annotation results.

9. A method for malicious code feature extraction and rule generation based on Transformer according to claim 8, characterized in that, The malicious code samples include: PE files, ELF files, script files, and source code; The conversion to a sequence format that the Transformer model can process includes: reading the binary file byte by byte and converting it into an integer sequence of 0-255; extracting the API call sequence of the program using a static analysis tool; and converting the extracted sequence into the token sequence format input to the Transformer model.

10. A malicious code feature extraction and rule generation system based on Transformer, characterized in that, include: The data input layer is used to obtain malicious code samples; The preprocessing layer is used to convert the acquired malicious code samples into a sequence format that can be processed by the Transformer model; The Transformer core layer is used to build pre-trained models; Classifier layer, used to train the classifier; The model interpretation layer is used to generate a comprehensive importance score for each input sequence by fusing the results of attention weight analysis, SHAP value calculation and gradient analysis as the model interpretation result, and to extract key feature fragments from the original malicious code sample based on the model interpretation result. The rule generation layer is used to convert the extracted key feature fragments into structured YARA rules and map them to their positions in the original binary file; Engine integration layer, used for integration with antivirus engines through standardized interfaces.

Images