An Adversarial Training Method for Fine-Grained Flow Obfuscation Based on Semantic Probability Reconstruction

Abstract

The present invention discloses an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, belonging to the technical field of adversarial training, including: extracting a control flow graph and optimizing it into a local functional block structure to generate functional attribute features of nodes; aggregating functional attribute features and topological relationships based on a graph attention mechanism to generate low-dimensional graph embeddings; inputting the low-dimensional graph embeddings, opcode sequences, and application programming interface call sequences into a large language model to learn the functional block call probability distribution of benign samples and lock abnormal node pairs; based on a reinforcement learning agent, searching for an optimal operation sequence including call transfer, node hiding, and semantic-irrelevant instruction injection; adjusting the control flow graph according to the optimal operation sequence to reconstruct an adversarial software that conforms to the constraints of the executable file format; and forming an attack-defense dynamic game framework by dynamically evaluating the attack success rate and semantic retention rate and feedback-optimizing the reinforcement learning agent. This method can improve the reliability and accuracy of malicious software defense.

Description

Technical Field

[0001] The present invention relates to an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, and belongs to the technical fields of malware attack and defense games and adversarial training. Background Art

[0002] With the sustainable development of computer technology, the proliferation of malware has become a serious security threat. Initially, the industry detected malware based on the unique identifier of malware samples, i.e., signatures, which usually consisted of specific code sequences, function calls, or characteristic patterns of malware. The defense system included suspicious software in the blacklist by frequently updating the database, thereby achieving the detection of malware. Due to its simplicity and effectiveness, signature-based malware detection technology is usually combined with other security technologies (such as firewalls, intrusion detection systems) and widely applied in multiple fields such as enterprise-level security protection, network security monitoring, and personal computer security protection. However, signature-based malware detection is easily circumvented by traditional obfuscation techniques such as compression, encryption, register reallocation, code virtualization, etc., and it is difficult to detect new types of malware, resulting in new challenges for malware detection.

[0003] In recent years, due to the rapid development of artificial intelligence technology, various machine learning (ML) and deep learning (DL) models have been applied to the field of malware detection. However, the latest research shows that learning-based malware detection methods are difficult to resist adversarial attacks. An adversarial attack refers to making a small perturbation to a malicious sample to form a sample that retains malicious attributes, inducing a learning-based model, especially a deep learning model, to make an incorrect decision, so that the adversarial sample can successfully bypass detection and achieve a malicious attack.

[0004] Existing adversarial attacks mainly include three types, namely injecting irrelevant application programming interface (API) calls, partially or globally manipulating the original bytes of malware, and manipulating the control flow graph (CFG) of malware. Research shows that for attack models against APIs and original bytes, adversarial features rather than executable files are generated, and they are strictly limited to specific detection systems, while attack models against CFG have good performance and scalability in combating malware detection. Further research shows that traditional CFG-based adversarial attacks aim to operate on nodes in the graph, i.e., basic blocks. The common method is code obfuscation and injecting semantic no operations (Nops). Only operating on nodes belongs to coarse-grained obfuscation and is difficult to bypass existing improved detection models. Summary of the Invention

[0005] The object of the present invention is to provide an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, which can reduce the limitations of traditional obfuscation techniques in avoiding advanced malware detection and improve the reliability and accuracy of malware defense.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The present invention provides an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, including:

[0008] Using a static analysis tool to parse the executable sample, extract the control flow graph and optimize it into a local functional block structure, and generate the functional attribute features of the nodes;

[0009] Based on the graph attention mechanism, aggregate the functional attribute features and topological relationships to generate a low-dimensional graph embedding;

[0010] Input the low-dimensional graph embedding, opcode sequence, and application programming interface call sequence into a large language model to learn the functional block call probability distribution of benign samples and lock abnormal node pairs;

[0011] Based on a reinforcement learning agent, with the attack success rate and semantic retention rate as the goals, search for the optimal operation sequence including call transfer, node hiding, and semantic-irrelevant instruction injection;

[0012] Adjust the control flow graph according to the optimal operation sequence and reconstruct an adversarial software that conforms to the executable file format constraints;

[0013] Through dynamic evaluation of the attack success rate and semantic retention rate, feedback and optimize the reinforcement learning agent to form an offensive and defensive dynamic game framework.

[0014] Furthermore, generating the functional attribute features of the nodes includes:

[0015] Extract the opcode sequence of the functional block and count the frequency to generate a numerical opcode feature vector;

[0016] Parse the application programming interface call sequence and generate a semantic-level application programming interface feature vector through a pre-trained language model;

[0017] Concatenate the opcode feature vector and the application programming interface feature vector to form the functional attribute features of the nodes.

[0018] Furthermore, generating the low-dimensional graph embedding includes: dynamically allocating weights to neighborhood nodes through the multi-head attention mechanism, combining multi-hop neighborhood aggregation and non-linear transformation to update node features, and generating a low-dimensional graph embedding including functional attribute features and topological relationships.

[0019] Further, learning the functional block call probability distribution includes: training a large language model through masked language modeling and sequence generation tasks, iteratively updating the call probability matrix between functional blocks until convergence, and storing the call probability matrix in a balanced binary tree for querying and updating.

[0020] Further, the update formula for the call probability matrix is:

[0021] ;

[0022] Among them, , represents the , th iteration of the call probability matrix from the th functional block to the th functional block , represents the iterative learning rate, represents the th functional block to the th functional block The actual number of calls, and the iteration termination condition is that the maximum change amount of the call probability matrix is less than the change amount threshold .

[0023] Further, the reward function of the reinforcement learning agent is:

[0024] ;

[0025] Among them, represents the reward value returned by the environment when the agent selects action at the th time step according to the state , , represent the escape detection probabilities at the , th time steps, and search for the optimal operation sequence by maximizing the cumulative reward value through the Q-function reinforcement learning algorithm.

[0026] Further, call transfer includes: inserting a jump instruction in the parent node instruction of the call exception node to point to the high-probability node, and adding an instruction to jump to the sub-address of the hidden node at the end of the high-probability node to ensure the coherence of the control flow logic.

[0027] Further, semantic-irrelevant instruction injection includes: injecting semantic-irrelevant instructions at the head of the functional block, and redirecting the execution flow to the original instruction sequence through jump instructions to ensure that the space occupied by the injected instructions conforms to the executable file format constraints.

[0028] Further, the calculation formula for the attack success rate is:

[0029] ;

[0030] where, represents the attack success rate, represents the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. represents the original malicious sample, represents the adversarial sample, represents the cardinality of the set, that is, the number of adversarial software that conforms to the executable file format constraints.

[0031] Further, the semantic retention rate is calculated by fusing the normalized difference degrees of the opcode sequence and the application programming interface call sequence. The calculation formula is:

[0032] ;

[0033] where, represents the semantic retention rate, represents the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. represents the original malicious sample, represents the adversarial sample, represents and the semantic similarity of, represents the cardinality of the set, that is, the number of adversarial software that conforms to the executable file format constraints.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] (1) Reducing the limitations of traditional obfuscation techniques: Traditional obfuscation techniques are easily recognized by advanced learning-based malware detection systems, while the fine-grained flow obfuscation technique of the present invention can better avoid existing detections, has strong generalization, and is applicable to multi-type malware detection scenarios.

[0036] (2) Evaluating the security risks in the black-box environment: The present invention uses a large language model to learn the call relationships between functional blocks in benign samples, combines a reinforcement learning agent to determine the optimal operation sequence, and evaluates the security risks of existing defense models in the black-box setting, which helps to improve the robustness of the detection system in unknown environments.

[0037] (3) Generating high-quality adversarial samples: The present invention complies with the executable file format constraints, efficiently retains the semantic functions of the original file, generates adversarial samples that can evade detection, and improves the adversarial training performance.

[0038] (4)Promote the development of the malware attack and defense game: Through the fine-grained control flow obfuscation technology, the present invention adjusts the control flow relationship of malware, making it difficult for existing detection systems to identify the original malicious attributes, which helps to promote the subsequent optimization of malware detection systems and improve the reliability and accuracy of malware defense. Description of the Drawings

[0039] Figure 1 It is a flowchart of the adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction provided by an embodiment of the present invention;

[0040] Figure 2 It is a flowchart of node embedding provided by an embodiment of the present invention;

[0041] Figure 3 It is a flowchart of learning call relationships provided by an embodiment of the present invention;

[0042] Figure 4 It is a schematic diagram of call transfer and node hiding provided by an embodiment of the present invention;

[0043] Figure 5 It is a schematic diagram of injecting semantically irrelevant instructions provided by an embodiment of the present invention;

[0044] Figure 6 It is a schematic diagram of the dynamic game framework provided by an embodiment of the present invention. Detailed Embodiments

[0045] The technical solutions of the present application will be further described in detail below in conjunction with the specific embodiments.

[0046] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application and should not be construed as a limitation of the present application. Without conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0047] An embodiment of the present application provides an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, including:

[0048] Use a static analysis tool to parse the executable sample, extract the control flow graph and optimize it into a local functional block structure, and generate the functional attribute features of the nodes;

[0049] Based on the graph attention mechanism, aggregate the functional attribute features and topological relationships to generate a low-dimensional graph embedding;

[0050] Input the low - dimensional graph embedding, opcode sequence, and application programming interface call sequence into the large - language model to learn the probability distribution of function block calls for benign samples and lock abnormal node pairs;

[0051] Based on the reinforcement learning agent, with the attack success rate and semantic retention rate as the goals, search for the optimal operation sequence that includes call transfer, node hiding, and semantic - irrelevant instruction injection;

[0052] Adjust the control flow graph according to the optimal operation sequence and reconstruct the adversarial software that conforms to the constraints of the executable file format;

[0053] Through dynamically evaluating the attack success rate and semantic retention rate, feedback to optimize the reinforcement learning agent to form an attack - defense dynamic game framework.

[0054] The embodiment of this application provides an adversarial training method for fine - grained flow obfuscation based on semantic probability reconstruction. First, extract the control flow graph and feature sequence of the software, transfer node attributes based on the graph attention network, and learn the function block call relationship of benign samples through the large - language model to realize the edge assignment of the graph structure of unlabeled samples, which helps to identify abnormal node pairs; second, use the reinforcement learning agent to search for the optimal operation sequence, synchronously operate nodes and edges to complete fine - grained flow obfuscation; finally, reconstruct the adversarial malware according to the operation sequence, abide by the constraints of the executable file format, evaluate the attack success rate and semantic retention rate and feedback them to the reinforcement learning agent to form a dynamic game system, verify the credibility of semantic probability reconstruction, help to supplement the control flow obfuscation technology, evaluate the security risks of existing detection models in the black - box setting, reduce the limitations of traditional obfuscation technologies when avoiding advanced malware detection, and improve the reliability and accuracy of malware defense.

[0055] The terms and constraints involved in the embodiments of this application include:

[0056] Control flow graph: The control flow graph is a graphical representation method used to show all possible execution paths in a program. The CFG consists of nodes and edges, where nodes represent basic blocks in the program, which are a sequence of continuously executed instructions; edges represent the transfer of control flow, indicating the call connection relationship between nodes and accurately reflecting the execution logic of the program. In the embodiments of this application, the control flow graph is updated by merging logical units to form function blocks for subsequent flow obfuscation and adversarial training.

[0057] Semantic probability: Semantic probability refers to the call probability between code function blocks. In the embodiments of this application, the call relationship between function blocks in benign samples is learned through the large - language model, and the call probability between function blocks in unlabeled samples is predicted, providing a theoretical basis for realizing fine - grained obfuscation of malware.

[0058] Control flow obfuscation: Control flow obfuscation is a software protection technique aimed at increasing the difficulty of understanding and analyzing a program by changing the control flow structure of the program, thereby enhancing the program's security. The control flow obfuscation in the embodiments of this application refers to a fine-grained flow obfuscation technique based on semantic probability reconstruction, involving operations such as adding, deleting, and modifying nodes and directed edges in the graph structure.

[0059] Adversarial attack: An adversarial attack refers to an attacker making tiny, usually imperceptible modifications to an input sample to mislead a machine learning model into making incorrect predictions or classifications. In the embodiments of this application, an adversarial attack specifically refers to performing fine-grained obfuscation on malware samples, that is, modifying nodes and edges while retaining the malicious attributes, enabling the malware to bypass machine learning-based malware detection systems and promoting the development of the attack-defense game.

[0060] Large language model: A large language model (LLM) is a deep learning model used to understand and generate natural language data. The model is usually trained on large-scale text data and can capture the complex patterns and structures of language. The large language model used in the embodiments of this application needs to process and understand the execution logic of software samples and predict the call probability of functional blocks in malicious samples.

[0061] Executable file format constraints: Format specifications that must be adhered to when reconstructing adversarial malware samples to ensure that the samples maintain the same semantics and performance as the original files. Any deviation from the standard format may cause the samples to malfunction. The format constraints ensure the legality of adversarial samples in terms of structure and function, enabling them to execute normally without raising suspicion.

[0062] The adversarial training method based on fine-grained flow obfuscation with semantic probability reconstruction provided by the embodiments of this application uses technologies such as large language models, control flow obfuscation, and adversarial attacks. The technical solutions described include the following objects:

[0063] Benign sample set : The benign sample set is a key data set in adversarial training and is used to construct and optimize the large language model. During the training process, the large language model analyzes the execution logic of benign samples by learning the call relationships between functional blocks in the graph, and assigns probability values to the directed edges for unlabeled samples. Edge assignment refers to assigning probability values to the edges (i.e., the call relationships between functional blocks) in the control flow graph based on the call probabilities predicted by the large language model, facilitating the subsequent identification of abnormal node pairs.

[0064] Malware samples :The malware sample is the main research object of the embodiments of this application. By obfuscating the control flow information of the sample, adversarial samples that retain malicious attributes are generated. This application performs complex and refined transformation operations on the nodes and edges in the graph, making it difficult to analyze and understand the behavior of the malware, which helps to improve the success rate of adversarial attacks.

[0065] Operation sequence : A series of operations decided by the reinforcement learning agent, which can effectively explore potential optimal operation sequences, including operations such as adding, deleting, and modifying nodes and call edges, namely call transfer, node hiding, and semantic injection. Through the above core operations, fine-grained flow obfuscation of the malware sample is achieved.

[0066] Adversarial software sample : The adversarial malware is an executable software obtained by reconstructing the training results. By mimicking the behavior patterns of the benign software sample set, it makes it difficult for the detection system to distinguish the differences between them and normal software, aiming to bypass learning-based malware detection and achieve high-quality adversarial training.

[0067] In a possible embodiment, as Figure 1 shown, the adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction specifically includes the following steps:

[0068] Step 1: Initialize the system and complete multi-dimensional feature extraction. In this embodiment, based on a static analysis tool, the assembly language executable sample is parsed to extract the control flow graph CFG, where the nodes represent basic blocks and the edges represent control flow relationships. On this basis, conditional branches are identified, and basic blocks are merged to form a local functional block optimization graph structure, the control flow graph structure is updated, the opcode syntax-level sequence and the API call semantic-level sequence are extracted, and after feature fusion, they are used as node functional attributes to provide a data basis for subsequent context analysis and adversarial training.

[0069] Step 1 specifically includes the following steps:

[0070] Step 1.1: Use a static analysis tool to extract the control flow graph from the binary benign sample set and identify the conditional branch merging logic unit to form local functional blocks, update the CFG graph structure and represent it as , where represents the set of nodes in the updated control flow graph, represents the set of edges in the updated control flow graph, represents the th node, that is, the th functional block, represents from node (that is, the th functional block) to node (i.e., the th functional block) of the directed call edge.

[0071] Step 1.2: Extract the opcode vector to obtain the features of the syntactic-level information. In this embodiment, the opcodes in the instruction set are mapped to the list , where , , …, represent the 1st, 2nd, …, th opcodes. According to the opcode sequence of the functional block , perform statistics, and represent the functional block as an array of the same size as the operation sequence , where , , …, represent the count values of the 1st, 2nd, …, th opcodes, and represents the numerical vector corresponding to the opcode feature, i.e., the opcode feature vector.

[0072] Step 1.3: Extract the API call vector to obtain the features of the semantic-level information. Analyze and extract the API call sequence of the functional block , and combine with the API documentation obtained from the official website. Use a bidirectional encoder model to generate the numerical vector corresponding to the API feature, i.e., the API feature vector: , where , , …, represent the corresponding numerical encodings in the call sequence.

[0073] Specifically, the opcode sequence reflects the underlying execution operations of the code, such as instructions like "mov", "add", etc.; the API call sequence reflects the interaction between the code and external systems or libraries, such as function calls like "CreateFileW", "ReadFile", etc.

[0074] The feature vectors extracted in Step 1.2 and Step 1.3 are shown in Table 1.

[0075] Table 1: API Call Sequence Encoding and Opcode Sequence Encoding.

[0076] .

[0077] Step 2: Implement context function transfer. To more accurately extract the function information of nodes, in this embodiment, the graph attention mechanism GAT is used to complete graph structure context learning, deeply analyze the control flow information, and generate a low-dimensional graph embedding containing functional attributes and topological relationships.

[0078] Step 2 specifically includes the following steps:

[0079] Step 2.1: Combine the opcode feature and the API call feature to update the graph to , where represents the adjacency matrix in the updated control flow graph, represents the functional attribute feature of the node, that is, the functional attribute feature of the function block, . To retain the features of two dimensions, is composed of and concatenated. Figure 1 In , , , represent the 1st, 2nd, 3rd, and 4th nodes, , , , represent the functional attribute features of the 1st, 2nd, 3rd, and 4th nodes.

[0080] Step 2.2: Construct context relationships to generate graph embeddings. For each function block in the graph, a recursive aggregation method is usually used to learn the context, where each node combines the feature vectors of its adjacent nodes to derive its own updated feature vector.

[0081] Specifically, as Figure 2 shown, based on feature fusion, the attention mechanism is applied to dynamically adjust the weights of different features to obtain the node function feature vector.

[0082] Through times of aggregation iteration, a node is represented by a feature vector that encapsulates the -hop neighborhood structure information. Formally, the th iteration is constructed as follows:

[0083] ;

[0084] where , represent the functional attribute feature vectors of the function block at the , th iterations, represents the functional attribute feature vector of the function block at the Feature vector of functional attributes at the i-th iteration, initialization , where is the feature vector of the functional attributes of the functional block , represents the feature vector aggregated from the adjacent nodes of the functional block at the i-th iteration, represents the adjacent nodes of represents feature fusion, represents aggregation. The choices of feature fusion and aggregation vary in different variants of the graph neural network.

[0085] Considering that the graph attention mechanism GAT adaptively assigns weights to adjacent nodes and updates the current node using the weighted sum value of adjacent nodes, which has the advantage of strong generalization for directed graphs, in this embodiment, GAT compresses the topological relationship of the graph structure into a low-dimensional vector, enabling the subsequent large language model to avoid parsing complex structures from scratch.

[0086] Specifically, the updated control flow graph structure and its node attributes are input into GAT to calculate the overall functional graph embedding of multi-dimensional features. During the iterative process of each layer of GAT, the attribute embedding of a node is passed to its adjacent nodes. With the help of the multi-head attention mechanism, each node can focus on more key adjacent nodes. For a given pair of adjacent nodes , the attention weight in the attention head and the layer structure can be calculated according to the following formula:

[0087] ;

[0088] where is the node embedding of the node at the layer, is the node hidden representation, is the learning parameter of the attention head at the layer, is the feed-forward neural network, is the activation function, the operator represents the concatenation operation, represents taken from the adjacent node of , represents the node embedding of the node at the layer, and the total number of attention heads is .

[0089] ​The update process of each node embedding based on the attention mechanism is represented as follows:

[0090] ;

[0091] Among them, 、 represent the functional attribute feature vectors of the functional block at the 、 layers, represents the functional attribute feature vector of the functional block at the layer, represents the attention learning parameter of the layer, represents the attention weight of the node pair , represents the non - linear transformation.

[0092] In this embodiment, the information features of the fused node and its adjacent nodes are updated based on the non - linear transformation to obtain , 、 represent the number of adjacent nodes of the nodes 、 .

[0093] Through the above steps, a low - dimensional graph embedding vector is generated for the node and is updated. This vector contains both the functional attributes of the node itself and its context relationship in the graph.

[0094] Step 3: Introduce a large - language model to learn the call relationships between functional blocks of benign samples. In this embodiment, the large - language model is used to analyze benign samples, understand their execution logic, learn and store the call relationships of benign samples. On this basis, the pre - trained large - language model predicts the call relationships of functional blocks of unlabeled samples. After edge assignment, abnormal node pairs are locked, which is convenient for subsequent targeted adversarial training. The specific process is as Figure 3 shown.

[0095] Step 3 specifically includes the following steps:

[0096] Step 3.1: Submit the multi - dimensional data obtained in the pre - processing stage to the large - language model. The LLM learns the logical relationships in benign samples, deeply understands the semantic relevance between different functional blocks and their call patterns. In this embodiment, the generated graph embedding vector, the original opcode sequence, and the API call sequence are combined to construct the input sequence as follows:

[0097] ;

[0098] Among them, is the original opcode sequence, is the original API call sequence.

[0099] Step 3.2: Learn the call relationships among benign samples. Use a labeled database to train the LLM, and the large language model is pre-trained using masked language modeling (MLM) and sequence generation tasks. In masked language modeling, randomly mask some elements in the opcode, API name, or graph embedding, and let the model predict the masked content. This embodiment is based on the Bidirectional Encoder Representations from Transformers (BERT) model to complete the training, and its cross-entropy loss function is:

[0100] ;

[0101] Among them, represents the parameters of the transformer encoder and output layer in BERT, represents the set of masked tokens in the training stage, represents the th masked token. In each self-attention layer in BERT, an input masked token updates its embedding through the weights of other connected token embeddings. The embedding of each token captures context-sensitive functional semantic information, which varies with the position and context.

[0102] Specifically, input "graph embedding [0.12, 0.34, [MASK], 0.56], opcode sequence {mov; cmp; [MASK]}, API call sequence {CreateFileW; [MASK]}", and predict the graph embedding, opcode, and API call information. Use accuracy and perplexity to evaluate the model's understanding ability of the input information, and improve the model's robustness through data augmentation (randomly shuffling the opcode order, replacing API calls with synonyms, etc.).

[0103] Step 3.3: Iteratively use the pre-trained model to update the call probability until the call relationship probability matrix of the functional blocks in the entire benign binary file sample library converges. Let represent the call probability between functional blocks, and construct the prompt as , where is the functional attribute feature vector of the functional block . Each iteration adjusts the probability value according to the model's prediction and the actual call data, gradually approaching the true call distribution.

[0104] Specifically, the graph embedding vector, opcode sequence, and API call sequence are concatenated into a text form according to certain rules, and the corresponding task description is added. For example, "The following is the relevant information of the functional block. Please understand its function and call relationship. Graph embedding [0.12, 0.34,..., 0.56], opcode sequence {mov; cmp; je}, API call sequence {CreateFileW; ReadFile}, and learn the call probability with other adjacent functional blocks."

[0105] Subsequently, in each iteration, the probability value will be adjusted according to the model's prediction and the actual call data, gradually approaching the true call probability distribution.

[0106] The update formula for the call probability matrix is as follows:

[0107] ;

[0108] where , represents the call probability matrix of the , th iteration from the th functional block to the th functional block , represents the iteration learning rate, represents the th functional block to the th functional block of the actual call count. The iteration termination condition is that the maximum change amount of the call probability matrix is less than the change amount threshold , that is .

[0109] During the model training process, the call probability loss function between nodes is denoted as , and the formula is as follows:

[0110] ;

[0111] where is the parameter vector used in model training, represents the probability of calling node given node and parameter vector in the training sample, and represent the number of samples and the number of nodes in a single sample respectively. On this basis, the model parameter is optimized as follows:

[0112] ;

[0113] Among them, represents the model learning rate, represents the gradient of the loss function with respect to the parameter , represents the element in is the regularization hyperparameter, and introducing the regularization term prevents the model from overfitting and improves the generalization ability of the model.

[0114] Step 3.4: Store the call probability. The call probability of the benign sample will be used as the basis for subsequent analysis and decision-making, providing the possibility of calls between each functional block in the program. Select an efficient data structure to store the final stable call probability matrix . In this embodiment, the balanced binary tree AVL tree is used for storage. Among them, the query operation is , and the update operation is , aiming to efficiently search and adjust the call probability matrix between functional blocks.

[0115] Step 4: Multi-objective programming to find the optimal operation sequence. Considering that the control flow transformation of malware is a discrete and high-dimensional space, this embodiment is based on a reinforcement learning (RL) agent to efficiently search for the optimal adversarial operation sequence. Through continuous attempts and adjustments, it aims to find an operation sequence that retains the core functions of the malware and effectively evades security detection at the lowest cost, providing key theoretical support for subsequent malware reconstruction.

[0116] Step 4 specifically includes the following steps:

[0117] Step 4.1: Evaluate the correlation between the node in the current sample and the node in the benign sample:

[0118] ;

[0119] Among them, and respectively represent the functional attribute feature sets in the current software and the benign sample set , represents the number of features. The numerator represents the number of similar features in the benign sample and the malicious sample, and the denominator represents the number of benign sample features. Finally, the comprehensive similarity of the current software and the benign sample data set is obtained. Design an evaluation threshold . A similarity lower than the threshold indicates a higher degree of abnormality, and adversarial obfuscation is required.

[0120] Step 4.2: Train the Q - learning network using the benign sample data in the memory buffer to determine the optimal obfuscation operation. Obtain the abnormal nodes by ascending the benign similarity of the sample nodes in ascending order to obtain abnormal nodes and submit them to the RL agent to complete the obfuscation work.

[0121] Since the Monte Carlo Tree Search (MCTS) allows, without prior information about the target system, that is, without in - depth understanding of the internal working mechanism of the target detection system, to find the optimal sequence to successfully avoid the target malware detection system through simulation in a large discrete space.

[0122] Specifically, select MCTS as the RL agent, input the updated CFG graph structure, and output the transformation sequence after multi - objective planning aiming to achieve the effect of evading detection with the minimum obfuscation cost:

[0123] .

[0124] This embodiment studies the black - box detection model in general cases, where represents the detection result of the alternative detector, the sequence limit length is , and for each selection of a functional block, perform iterative calculations of the optimization algorithm for times to find the functional block with the highest reward value for operation, and synchronously update the selected basic block and the corresponding operation to the sequence. For each of the

[0125] times, use the Monte Carlo tree to calculate the search sequence, and assign values to each participating functional block after obtaining the reward function of the adversarial sample. Represent the control flow graph of the malicious sample after update as the initial state , and for each time step, the RL agent selects an action , where the actions include Nops semantic injection, call transfer, and node hiding of the current node and its neighborhood nodes. Among them, the Nops instruction is a special instruction that does not perform any operation and is often used for code obfuscation. In this embodiment, the RL agent calculates the decision reward value , where represents the current graph state at the current moment, and represents the operation performed on the selected functional block. Calculate the probability of the evasion model as , where

[0126] The reward function of the reinforcement learning agent is:

[0127] ;

[0128] Among them, represents the -th time step when the agent selects an action according to the state The reward value returned by the environment, and represents the -th evasion detection probability at the time step. The optimal operation sequence is searched by maximizing the cumulative reward value through the Q-function reinforcement learning algorithm.

[0129] Based on the reward function, the Q-function is defined as: the expected return that can be obtained by taking the action under the state . Let be the expected function, and take the constant as the discount factor. The formula is as follows:

[0130] .

[0131] Use the Q-function reinforcement learning (Q-learning) algorithm to learn the optimal policy and iteratively update the Q-function. Among them, the complexity of selecting instruction injection is , is the number of semantic Nops applied in this embodiment. Once the injected instruction reaches the constraint or successfully evades, the reinforcement learning process will stop and feedback the current optimal operation sequence.

[0132] In MCTS, each node represents a state , and the obfuscation policy is the action . In this embodiment, the obfuscation policy includes semantic injection, call transfer, and node hiding. The probability of calculating the evasion model is , represents the target label. If the probability increases, it is 1, otherwise it is 0. The reward value is:

[0133] .

[0134] In this embodiment, the abnormal node sorting information is obtained according to the node similarity and used as the prior knowledge to initialize the node priority of the MCTS tree. In the MCTS selection stage, the upper confidence bound formula (Upper Confidence Bound, UCB) is optimized, considering the node visit times, reward value, and abnormal sorting to balance exploration and exploitation. The specific formula is:

[0135] ;

[0136] Among them, represents the node​ The score in the selection phase, the first item represents the average reward value, represents the node reward value, represents the node the number of visits; the second item is the exploration term, which encourages MCTS to select nodes with fewer visits, represents the node parent node of the number of visits, the constant term controls the exploration intensity, adjusted within the range of [0.5, 2.0] through grid search, with the default value being ; the third item is a reference item newly added in this embodiment, represents the outlier of the node, the constant term controls the priority of abnormal nodes, dynamically scaled according to the similarity distribution, with the default value being . Secondly, based on select the node with the highest score in turn to expand, simulate, and backtrack, quickly and efficiently identifying and processing abnormal nodes.

[0137] Step 4.3: In this embodiment, one of the actions is to call transfer and node hiding. First, query the nodes with high similarity to the abnormal node in the neighborhood. If there are similar nodes, perform the hiding and transfer operations. Among them, the model quantifies the similarity between nodes within the same sample based on the k-Nearest Neighbors (KNN) algorithm, and determines the node similarity by calculating the distance between feature vectors:

[0138] ;

[0139] Among them, , represent the Opcode operation code similarity and API call similarity of the node pair , , represent the distance between the Opcode operation code vectors of the node pair , the node pair , , represent the distance between the API call vectors of the node pair , the node pair , represents the number of nodes.

[0140] According to the Euclidean distance of the node features, select the node in the neighborhood that is closest to it , lock the nodes with abnormal call relationships for obfuscation. For specific operations, see Step 5.2.

[0141] Step 4.4: If there are no similar nodes in the neighborhood of the abnormal node, select the action as "injecting common Nops instructions in the benign sample set". The instructions include but are not limited to add / sub, push / pop, xor, mov, test, and, cmp, or, etc. The injection location is selected at the head of the function block.

[0142] Specifically, inject Nops instructions at the head of the current node. After the injection of Nops instructions is completed, use instructions to complete the logical jump. See Step 5.3 for details, and it is required to meet the executable constraints.

[0143] Step 5: Implement adversarial software reconstruction based on executable file constraints . In this embodiment, fine-grained obfuscation is implemented for the control flow, and on this basis, an executable adversarial software is reconstructed. During the reconstruction process, the nodes and edges are finely adjusted and optimized, and instructions are used to implement function jumps and calls. The model needs to effectively evade existing security protection while retaining the core functions of the original malware.

[0144] Step 5 specifically includes the following steps:

[0145] Step 5.1: Define the node selection strategy and the obfuscation action set. According to the adjacency matrix , based on the node function attributes find the node with the highest similarity, that is the node with the closest Euclidean feature distance within its adjacency range . Judge and whether the similarity exceeds the threshold . If it exceeds the threshold, select the action as call transfer and node hiding; otherwise, inject Nops instructions that are semantically irrelevant to the node.

[0146] Step 5.2: Call transfer and node hiding. Denote and the one with a small call probability in , and the one with a large call probability is denoted as . In this embodiment, is implemented for call transfer and hiding.

[0147] Specifically, for the parent node of the call , insert instructions before its final jump / call instruction to point to , and complete the abnormal node The called relationship is transferred. Complete the polymorphic operation of input and output and add the call relationship label. The input and output parts complete the branch jump according to the label: if the call The parent node is , then in Insert before the last instruction of the block Instruction, jump to the hidden node Child nodes of Based on the above, complete the nodes within the neighborhood , that is, to realize the node "deletion" and edge "transfer" operations.

[0148] Specifically, if the functional attributes of node 002 and node 003 are similar, and the probability of node 003 being called is small, then , ,The specific call transfer is shown in Table 2.

[0149] Table 2: Instructions for calling transfer operations.

[0150] .

[0151] Among them, When a node completes a polymorphic input and output operation, it must retain the corresponding Call relationship label, that is The parent and child node numbers of the nodes. Taking the call relationship "001→003→004" as an example, node 003 is hidden due to an exception, and its function is completed by the polymorphic node 002. At this time, the corresponding polymorphic label is 002 (1-4), where 1 represents the parent node 001, 4 represents the child node 004, and so on. After clarifying the node polymorphic label, you need to Node head and tail injection correspondence Instruction, where 002 (1-4)_last represents the last instruction of the 002 node pair (1-4) label after the polymorphism, completing the executable. The specific visualization process is as follows Figure 4 shown.

[0152] Step 5.3: Inject semantically irrelevant instructions Nops into the node. First, and Respectively Specifically, the size and address of the free space in the ".text" section of the current node The first address is , the length of the first instruction is , the length of the second instruction is In this embodiment, the node The second instruction is The command is updated to ,make The first instruction at the address is the original second instruction the content at that place, and then Nops instructions are injected for obfuscation. Finally, at the end of the injected instructions, an instruction is added to jump to the third instruction at that place to ensure that the space size required for all injected instructions does not exceed , meeting the executable conditions and constraint specifications. The specific visualization process is as Figure 5 shown.

[0153] Step 6: Evaluate the performance of adversarial training. In the fields of security research and malware attack and defense, important metrics for measuring the performance of adversarial training include the attack success rate ASR and the semantic retention rate SPR. The attack success rate reflects the ability of malware to bypass security detection, while the semantic retention rate measures the degree of retention of the core functions of the software after adversarial training. In this embodiment, the above evaluation metrics are optimized, and the performance evaluation is fed back to the reinforcement learning agent to build an attack and defense dynamic game framework.

[0154] Step 6 specifically includes the following steps:

[0155] Step 6.1: Calculate the attack success rate ASR. ASR is the most commonly used metric for evaluating adversarial attacks, defined as the proportion of malware that successfully bypasses the detection of the target system among all malware. The calculation formula is as follows:

[0156] ;

[0157] where, represents the attack success rate, represents the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. represents the original malicious sample, represents the adversarial sample, represents the cardinality of the set, that is, the number of adversarial software that conforms to the executable file format constraints.

[0158] Step 6.2: Calculate the semantic retention rate SPR. The existing semantic retention rate only considers the API sequence. In this embodiment, the semantic retention is measured by integrating the API sequence and the Opcode operation code sequence. Among them, the difference between the adversarial software and the original software is calculated based on the API call sequence, denoted as , and the calculation formula is as follows:

[0159] ;

[0160] where, , represent malware samples and the API sequence features in the adversarial software sample denote the API sequence features in the malware sample and the length of the API sequence features in the adversarial software sample denote the difference between and . Similarly, the difference degree based on the Opcode sequence is denoted as and the calculation formula is as follows: ;

[0161] where denote the Opcode sequence features in the malware sample

[0162] and

[0163] denote the length of the Opcode sequence features in the malware sample and denote the difference between and . Considering comprehensively, the weight is used to fuse the API sequence difference degree between the adversarial malware and the original malware and the Opcode sequence difference degree to obtain the sample difference degree . On this basis, the semantic similarity is calculated, and the formula is as follows: ;

[0164] where denotes the evaluation threshold determined by the average value of the sample difference degree. Considering that the partial random calls of APIs in the sandbox will lead to of the same software, in this embodiment, the same sandbox is used twice to analyze all original malware samples, and the average value of the difference degrees of all samples is calculated to determine . Finally, the semantic preservation rate SPR is defined as the proportion of adversarial malware that retains the original semantics among the adversarial malware that successfully bypasses detection, and the calculation formula is as follows: .

[0165] Based on this, the semantic similarity is calculated, and the formula is as follows:

[0166] ;

[0167] where denotes the evaluation threshold determined by the average value of the sample difference degree. Considering that the partial random calls of APIs in the sandbox will lead to

[0168] of the same software, in this embodiment, the same sandbox is used twice to analyze all original malware samples, and the average value of the difference degrees of all samples is calculated to determine . Finally, the semantic preservation rate SPR is defined as the proportion of adversarial malware that retains the original semantics among the adversarial malware that successfully bypasses detection, and the calculation formula is as follows: ;

[0169] ;

[0170] Among them, represents the semantic retention rate, represents the detection result of the malware detector for the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. represents the original malicious sample, represents the adversarial sample, represents and the semantic similarity of, represents the cardinality of the set, that is, the number of adversarial softwares that conform to the executable file format constraints.

[0171] Step 6.3: Feed the actual evaluation result back to the reinforcement learning RL agent, compare and analyze the prediction result and the actual result to update the alternative detector, improve the reward function in the reinforcement learning agent, continuously optimize and train the simulation process, construct a dynamic game framework, and improve the performance of the adversarial training system.

[0172] Specifically, feed the actual evaluation result back to the Monte Carlo tree MCTS, compare and analyze the prediction result and the actual result to update the alternative detector, improve the reward function in the reinforcement learning agent, continuously optimize and train the simulation process, construct a dynamic game framework as shown in Figure 6 to improve the performance of the adversarial training system.

[0173] The above is only the preferred implementation manner of this application. It should be noted that for those of ordinary skill in the art of this technology, without departing from the technical principle of this application, several improvements and deformations can still be made, and these improvements and deformations should also be regarded as the protection scope of this application.

Claims

1. An adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, characterized in that: include: Use static analysis tools to parse executable samples, extract control flow graphs and optimize them into local function block structures, and generate functional attribute characteristics of nodes; Aggregate functional attribute features and topological relationships based on graph attention mechanism to generate low-dimensional graph embedding; The low-dimensional graph embedding, opcode sequence and API call sequence are input into the large language model to learn the function block call probability distribution of benign samples and lock the abnormal node pairs; Based on reinforcement learning agent, the optimal operation sequence including call transfer, node hiding and semantic irrelevant instruction injection is searched with the attack success rate and semantic preservation rate as the target; Adjust the control flow graph according to the optimal operation sequence and reconstruct the adversarial software to meet the constraints of the executable file format; By dynamically evaluating the attack success rate and semantic retention rate, feedback is provided to optimize the reinforcement learning agent, forming a dynamic attack and defense game framework; The functional attribute characteristics of the generated node include: Extract the operation code sequence of the function block and generate a numerical operation code feature vector by counting the frequency; Parse the API call sequence and generate semantic-level API feature vectors through a pre-trained language model; Concatenate the operation code feature vector and the application programming interface feature vector to form the functional attribute feature of the node; The call transfer includes: inserting a jump instruction to point to a high-probability node in the parent node instruction of the call exception node, and adding an instruction to jump to the sub-address of the hidden node at the end of the high-probability node to ensure the logical coherence of the control flow.

2. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: Generating low-dimensional graph embeddings includes: dynamically allocating weights of neighborhood nodes through a multi-head attention mechanism, combining multi-hop neighborhood aggregation and nonlinear transformation to update node features, and generating low-dimensional graph embeddings that include functional attribute features and topological relationships.

3. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: Learning the function block call probability distribution of benign samples includes: training a large language model through masked language modeling and sequence generation tasks, iteratively updating the call probability matrix between function blocks until convergence, and storing the call probability matrix in a balanced binary tree for query and update.

4. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 3 is characterized in that: The update formula of the call probability matrix is: ； in, , Indicates , The first iteration Function Blocks To Function Blocks The call probability matrix, represents the iterative learning rate, Indicates Function Blocks To Function Blocks The actual number of calls, the iteration termination condition is that the maximum change of the calling probability matrix is ​​less than the change threshold .

5. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: The reward function for the reinforcement learning agent is: ； in, Indicates The agent is based on the state Select Action The reward value returned by the environment, , Indicates , The probability of escaping detection in time steps is calculated, and the optimal operation sequence is searched by maximizing the cumulative reward value through the Q-function reinforcement learning algorithm.

6. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: Semantically irrelevant instruction injection includes: injecting semantically irrelevant instructions into the header of the function block, and redirecting the execution flow to the original instruction sequence through jump instructions, ensuring that the space occupied by the injected instructions complies with the executable file format constraints.

7. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: The calculation formula for the attack success rate is: ； in, Indicates the attack success rate, Indicates the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. Represents the original malicious sample, represents adversarial examples, Represents the cardinality of the set, i.e., the number of adversarial software that conform to the executable file format constraints.

8. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: The semantic retention rate is calculated by fusing the normalized difference between the opcode sequence and the API call sequence, and the calculation formula is: ； in, represents the semantic retention rate, Indicates the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. Represents the original malicious sample, represents adversarial examples, express and The semantic similarity of Represents the cardinality of the set, i.e., the number of adversarial software that conform to the executable file format constraints.

Images