Semantic probability reconstruction-based adversarial training method for fine-grained flow confusion

Abstract

The invention discloses an adversarial training method for fine-grained flow confusion based on semantic probability reconstruction, which belongs to the technical field of adversarial training, and comprises the following steps: extracting a control flow graph, optimizing the control flow graph into a local function block structure, and generating functional attribute features of nodes; on the basis of a graph attention mechanism, aggregating functional attribute features and a topological relation, and generating low-dimensional graph embedding; the low-dimensional graph is embedded, an operation code sequence and an application programming interface calling sequence are input into a large language model, function block calling probability distribution of a benign sample is learned, and abnormal node pairs are locked; based on a reinforcement learning agent, searching an optimal operation sequence including call transfer, node hiding and semantic independent instruction injection; adjusting the control flow diagram according to the optimal operation sequence, and reconstructing antagonistic software conforming to executable file format constraints; and an attack success rate and a semantic retention rate are dynamically evaluated, and an optimization reinforcement learning agent is fed back, so that an attack and defense dynamic game framework is formed. According to the method, the malicious software defense reliability and accuracy can be improved.

Description

Technical Field

[0001] The invention relates to an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, and belongs to the technical field of malware attack and defense game and adversarial training. Background Art

[0002] With the sustainable development of computer technology, the proliferation of malware has become a serious security threat. Initially, the industry detected malware based on unique identifiers of malware samples, namely signatures. These signatures usually consist of specific code sequences, function calls or characteristic patterns of malware. The defense system blacklists suspicious software by frequently updating the database, thereby realizing the detection of malware. Signature-based malware detection technology, due to its simplicity and effectiveness, is usually combined with other security technologies (such as firewalls, intrusion detection systems), and is widely used in enterprise-level security protection, network security monitoring, and personal computer security protection. However, signature-based malware detection can be easily circumvented by traditional obfuscation technologies such as compression, encryption, register reallocation, code virtualization, etc. At the same time, it is difficult to detect new malware, resulting in new challenges for malware detection.

[0003] In recent years, due to the rapid development of artificial intelligence technology, various machine learning (ML) and deep learning (DL) models have been applied to the field of malware detection, but the latest research shows that learning-based malware detection methods are difficult to resist adversarial attacks. Adversarial attacks refer to making small perturbations to malicious samples to form samples that retain malicious properties, inducing learning-based models, especially deep learning models, to make wrong decisions, so that adversarial samples can successfully bypass detection and carry out malicious attacks.

[0004] There are three main types of existing adversarial attacks, namely injecting irrelevant Application Program Interface (API) calls, partially or globally manipulating the raw bytes of malware, and manipulating the control flow graph (CFG) of malware. Studies have shown that attack models targeting APIs and raw bytes generate adversarial features rather than executable files, and are strictly limited to specific detection systems, while attack models targeting CFG have good performance and scalability when fighting against malware detection. Further research shows that traditional CFG-based adversarial attacks aim to manipulate nodes in the graph, namely basic blocks. Common methods are code obfuscation and injecting semantic null instructions (NoOperations, Nops). Manipulating nodes alone is coarse-grained obfuscation, which is difficult to bypass existing improved detection models. Summary of the invention

[0005] The purpose of the present invention is to provide an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, which can reduce the limitations of traditional obfuscation technology in evading advanced malware detection and improve the reliability and accuracy of malware defense.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: The present invention provides an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, comprising: Use static analysis tools to parse executable samples, extract control flow graphs and optimize them into local function block structures, and generate functional attribute characteristics of nodes; Aggregate functional attribute features and topological relationships based on graph attention mechanism to generate low-dimensional graph embedding; The low-dimensional graph embedding, opcode sequence and API call sequence are input into the large language model to learn the function block call probability distribution of benign samples and lock the abnormal node pairs; Based on reinforcement learning agent, the optimal operation sequence including call transfer, node hiding and semantic irrelevant instruction injection is searched with the attack success rate and semantic preservation rate as the target; Adjust the control flow graph according to the optimal operation sequence and reconstruct the adversarial software that complies with the executable file format constraints; By dynamically evaluating the attack success rate and semantic retention rate, feedback is provided to optimize the reinforcement learning agent and form a dynamic attack and defense game framework.

[0007] Furthermore, the functional attribute characteristics of the generated node include: Extract the operation code sequence of the function block and generate a numerical operation code feature vector by counting the frequency; Parse the API call sequence and generate semantic-level API feature vectors through a pre-trained language model; The operation code feature vector and the application programming interface feature vector are concatenated to form the functional attribute feature of the node.

[0008] Furthermore, generating low-dimensional graph embeddings includes: dynamically allocating weights of neighborhood nodes through a multi-head attention mechanism, combining multi-hop neighborhood aggregation and nonlinear transformation to update node features, and generating low-dimensional graph embeddings that include functional attribute features and topological relationships.

[0009] Furthermore, learning the function block call probability distribution of benign samples includes: training a large language model through masked language modeling and sequence generation tasks, iteratively updating the call probability matrix between function blocks until convergence, and storing the call probability matrix in a balanced binary tree for query and update.

[0010] Furthermore, the update formula of the call probability matrix is: ; in, , Indicates , The first iteration Function Blocks To Function Blocks The call probability matrix, represents the iterative learning rate, Indicates Function Blocks To Function Blocks The actual number of calls, the iteration termination condition is that the maximum change of the calling probability matrix is ​​less than the change threshold .

[0011] Furthermore, the reward function of the reinforcement learning agent is: ; in, Indicates The agent is based on the state Select Action The reward value returned by the environment, , Indicates , The probability of escaping detection in time steps is calculated, and the optimal operation sequence is searched by maximizing the cumulative reward value through the Q-function reinforcement learning algorithm.

[0012] Furthermore, the call transfer includes: inserting a jump instruction pointing to a high-probability node in the parent node instruction of the call exception node, and adding an instruction to jump to the sub-address of the hidden node at the end of the high-probability node to ensure the logical coherence of the control flow.

[0013] Furthermore, the semantically irrelevant instruction injection includes: injecting semantically irrelevant instructions into the header of the function block, and redirecting the execution flow to the original instruction sequence through a jump instruction, ensuring that the space occupied by the injected instruction complies with the executable file format constraints.

[0014] Furthermore, the calculation formula for the attack success rate is: ; in, Indicates the attack success rate, Indicates the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. Represents the original malicious sample, represents adversarial examples, Represents the cardinality of the set, i.e., the number of adversarial software that conform to the executable file format constraints.

[0015] Furthermore, the semantic retention rate is calculated by fusing the normalized difference between the opcode sequence and the API call sequence, and the calculation formula is: ; in, represents the semantic retention rate, Indicates the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. Represents the original malicious sample, represents adversarial examples, express and The semantic similarity of Represents the cardinality of the set, i.e., the number of adversarial software that conform to the executable file format constraints.

[0016] Compared with the prior art, the present invention has the following beneficial effects: (1) Reducing the limitations of traditional obfuscation technology: Traditional obfuscation technology is easily identified by advanced learning-based malware detection systems, while the fine-grained flow obfuscation technology of the present invention can better evade existing detection, has strong generalization, and is suitable for multiple types of malware detection scenarios.

[0017] (2) Evaluating security risks in a black-box environment: This paper uses a large language model to learn the calling relationship between functional blocks in benign samples, combines it with a reinforcement learning agent to determine the optimal operation sequence, and evaluates the security risks of existing defense models in a black-box setting, which helps to improve the robustness of the detection system in an unknown environment.

[0018] (3) Generate high-quality adversarial samples: The present invention complies with the constraints of executable file formats, efficiently preserves the semantic functions of the original files, generates adversarial samples that can evade detection, and improves adversarial training performance.

[0019] (4) Promoting the development of malware attack and defense game: The present invention uses fine-grained control flow obfuscation technology to adjust the control flow relationship of malware, making it difficult for existing detection systems to identify the original malicious attributes, which helps promote the subsequent optimization of malware detection systems and improve the reliability and accuracy of malware defense. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction provided by an embodiment of the present invention; Figure 2 is a node embedding flow chart provided by an embodiment of the present invention; Figure 3 is a flow chart of a learning call relationship provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of call transfer and node hiding provided by an embodiment of the present invention; Figure 5 is a schematic diagram of semantically irrelevant instruction injection provided by an embodiment of the present invention; Figure 6 It is a schematic diagram of a dynamic game framework provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present application is further described in detail below in conjunction with specific implementation methods.

[0022] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and should not be construed as limitations on the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments may be combined with each other.

[0023] The embodiment of the present application provides an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, including: Use static analysis tools to parse executable samples, extract control flow graphs and optimize them into local function block structures, and generate functional attribute characteristics of nodes; Aggregate functional attribute features and topological relationships based on graph attention mechanism to generate low-dimensional graph embedding; The low-dimensional graph embedding, opcode sequence and API call sequence are input into the large language model to learn the function block call probability distribution of benign samples and lock the abnormal node pairs; Based on reinforcement learning agent, the optimal operation sequence including call transfer, node hiding and semantic irrelevant instruction injection is searched with the attack success rate and semantic preservation rate as the target; Adjust the control flow graph according to the optimal operation sequence and reconstruct the adversarial software that complies with the executable file format constraints; By dynamically evaluating the attack success rate and semantic retention rate, feedback is provided to optimize the reinforcement learning agent and form a dynamic attack and defense game framework.

[0024] The embodiment of the present application provides an adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction. First, the control flow graph and feature sequence of the software are extracted, node attributes are transferred based on the graph attention network, and the function block call relationship of the benign sample is learned through the large language model to realize the graph structure edge assignment of the unlabeled sample, which is helpful to identify abnormal node pairs; secondly, the reinforcement learning agent is used to search for the optimal operation sequence, and the operation nodes and edges are synchronized to complete the fine-grained flow obfuscation; finally, the adversarial malware is reconstructed according to the operation sequence, complying with the constraints of the executable file format, evaluating the attack success rate and semantic retention rate and feeding back to the reinforcement learning agent, forming a dynamic game system, verifying the credibility of the semantic probability reconstruction, helping to supplement the control flow obfuscation technology, evaluating the security risks of the existing detection model under the black box setting, reducing the limitations of traditional obfuscation technology in evading advanced malware detection, and improving the reliability and accuracy of malware defense.

[0025] The terms and constraints involved in the embodiments of this application include: Control flow graph: A control flow graph is a graphical representation method used to display all possible execution paths in a program. CFG consists of nodes and edges, where nodes represent basic blocks in the program, which are a sequence of instructions that are executed continuously; edges represent the transfer of control flow, indicating the connection relationship between node calls, and accurately reflecting the execution logic of the program. In an embodiment of the present application, the control flow graph is updated by merging logical units to form functional blocks for subsequent flow obfuscation and adversarial training.

[0026] Semantic probability: Semantic probability refers to the calling probability between code function blocks. In the embodiments of the present application, a large language model is used to learn the calling relationship between function blocks in benign samples, and the calling probability between function blocks in unlabeled samples is predicted, providing a theoretical basis for achieving fine-grained obfuscation of malware.

[0027] Control flow obfuscation: Control flow obfuscation is a software protection technology that aims to increase the difficulty of understanding and analyzing a program by changing its control flow structure, thereby improving the security of the program. The control flow obfuscation in the embodiments of the present application refers to a fine-grained flow obfuscation technology based on semantic probability reconstruction, involving operations such as adding, deleting, and modifying nodes and directed edges in a graph structure.

[0028] Adversarial attack: Adversarial attack refers to the attacker making small, usually imperceptible modifications to the input sample to mislead the machine learning model to make incorrect predictions or classifications. In the embodiment of the present application, adversarial attack specifically refers to fine-grained obfuscation of malware samples, that is, modifying nodes and edges while retaining malicious attributes, so that malware can bypass the machine learning-based malware detection system and promote the development of attack and defense games.

[0029] Large Language Model: A Large Language Model (LLM) is a deep learning model used to understand and generate natural language data. The model is usually trained on large-scale text data and can capture the complex patterns and structures of language. The large language model used in the embodiments of the present application needs to process and understand the execution logic of the software sample and predict the probability of function block calls in malicious samples.

[0030] Executable file format constraints: Format specifications that must be followed when reconstructing adversarial malware samples to ensure that the samples maintain the same semantics and performance as the original files. Any deviation from the standard format may cause the sample to not function properly. Format constraints ensure the legality of the structure and function of the adversarial sample, allowing it to be executed normally without arousing suspicion.

[0031] The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction provided in the embodiment of the present application uses technologies such as large language model, control flow obfuscation, adversarial attack, etc. The described technical solution includes the following objects: Benign sample set :The benign sample set is a key data set in adversarial training and is used to build and optimize the large language model. During the training process, the large language model analyzes the execution logic of the benign samples by learning the call relationship between the function blocks in the graph, and implements directed edge assignment for the unlabeled samples. Edge assignment refers to assigning probability values ​​to the edges in the control flow graph (i.e., the call relationship between function blocks) based on the call probability predicted by the large language model, which facilitates the subsequent identification of abnormal node pairs.

[0032] Malware samples Malware samples are the main research object of the embodiments of this application. By obfuscating the sample control flow information, adversarial samples with malicious attributes are generated. This application performs complex and sophisticated transformation operations on the nodes and edges in the graph, making the behavior of the malware difficult to analyze and understand, which helps to improve the success rate of adversarial attacks.

[0033] Operation sequence : A series of operations decided by the reinforcement learning agent can effectively explore the potential optimal operation sequence, including operations such as adding, deleting, and modifying nodes and call edges, namely call transfer, node hiding, and semantic injection. Through the above core operations, fine-grained flow obfuscation of malware samples is achieved.

[0034] Adversarial software samples : Adversarial malware is executable software obtained by reconstructing training results. It imitates the behavior patterns of benign software sample sets, making it difficult for detection systems to distinguish the differences between them and normal software. It aims to bypass learning-based malware detection and achieve high-quality adversarial training.

[0035] In one possible embodiment, Figure 1 As shown in FIG. 1 , the adversarial training method of fine-grained flow obfuscation based on semantic probability reconstruction specifically includes the following steps: Step 1: Initialize the system and complete multi-dimensional feature extraction. This embodiment parses the assembly language executable sample based on the static analysis tool and extracts the control flow graph CFG, where the nodes represent basic blocks and the edges represent control flow relationships. On this basis, the conditional branches are identified, the basic blocks are merged to form a local function block optimization graph structure, the control flow graph structure is updated, the opcode Opcode syntax-level sequence and the API call semantic-level sequence are extracted, and the feature fusion is used as the node function attribute to provide a data basis for subsequent context analysis and adversarial training.

[0036] Step 1 specifically includes the following steps: Step 1.1: Use static analysis tools to extract samples from a benign set of binaries Extract the control flow graph, identify the conditional branch and merge the logic unit, form a local function block, update the CFG graph structure and express it as ,in, represents the set of nodes in the control flow graph after the update, represents the edge set in the control flow graph after update, Indicates Node, that is, Function blocks, Represents a slave node (i.e. function blocks) to nodes (i.e. function blocks).

[0037] Step 1.2: Extract the opcode vector to obtain the features of the syntax level information. opcodes are mapped to lists Among them, , , …, Indicates the 1st, 2nd, ..., Operation code, according to the function block The opcode sequence Perform statistics and convert the function blocks Represented as a sequence of AND operations Arrays of the same size ,in, , , …, Indicates the 1st, 2nd, ..., The count value of the opcodes, Represents the numerical vector corresponding to the opcode feature, that is, the opcode feature vector.

[0038] Step 1.3: Extract API call vectors to obtain features of semantic level information. Analyze and extract functional blocks API call sequence , combined with the API documentation obtained from the official website, the bidirectional encoder model is used to generate the numerical vector corresponding to the API features, namely the API feature vector: ,in, , , …, Indicates the corresponding numerical code in the call sequence.

[0039] Specifically, the opcode sequence reflects the underlying execution operations of the code, such as instructions such as "mov" and "add", while the API call sequence reflects the interaction between the code and external systems or libraries, such as function calls such as "CreateFileW" and "ReadFile".

[0040] The feature vectors extracted in step 1.2 and step 1.3 are shown in Table 1.

[0041] Table 1: API call sequence encoding and opcode sequence encoding.

[0042] .

[0043] Step 2: Implement contextual function transfer. In order to more accurately extract the functional information of the node, this embodiment uses the graph attention mechanism GAT to complete graph structure context learning, deeply analyze the control flow information, and generate a low-dimensional graph embedding containing functional attributes and topological relationships.

[0044] Step 2 specifically includes the following steps: Step 2.1: Combine the opcode features with the API call features and update the graph to ,in, represents the adjacency matrix in the updated control flow graph, Represents the functional attribute characteristics of the node, that is, the functional attribute characteristics of the function block, To preserve the features of two dimensions, Depend on and Stitched together. Figure 1 middle, , , , Indicates the 1st, 2nd, 3rd, and 4th nodes, , , , Represents the functional attribute characteristics of the 1st, 2nd, 3rd, and 4th nodes.

[0045] Step 2.2: Contextual relationship construction, generating graph embedding. For each functional block in the graph, a recursive aggregation method is usually used to learn the context, where each node combines the feature vectors of its neighbors to derive its own updated feature vector.

[0046] Specifically, Figure 2 As shown in the figure, based on feature fusion, the attention mechanism is applied to dynamically adjust the weights of different features to obtain the node function feature vector.

[0047] pass In each aggregation iteration, a node is represented by a feature vector that encapsulates its The structural information in the jump neighborhood. Formally, The iteration structure is as follows: ; in, , Indicates function block In the , The functional attribute feature vector at the iteration, Indicates function block In the The feature vector of the functional attributes at the iteration is initialized ,in, It is a function block The feature vector of the functional attributes, Indicated in The first iteration from the function block The feature vector of the adjacent node aggregation, express The adjacent nodes of represents feature fusion, The choice of representation aggregation, feature fusion, and aggregation varies in different variants of graph neural networks.

[0048] Considering that the graph attention mechanism GAT adaptively assigns weights to adjacent nodes and uses the weighted sum of adjacent nodes to update the current node, it has a strong generalization advantage for directed graphs. This embodiment compresses the topological relationship of the graph structure into a low-dimensional vector through GAT, so that the subsequent large language model does not need to parse the complex structure from scratch.

[0049] Specifically, the updated control flow graph structure and its node attributes are input into GAT to calculate the embedding of the multi-dimensional feature overall functional graph. In the iteration process of each layer of GAT, the attribute embedding of the node is passed to its adjacent nodes. With the help of the multi-head attention mechanism, each node can pay attention to more key adjacent nodes. For a given adjacent node pair , in the attention head and Attention weights of the layer structure It can be calculated according to the following formula: ; in, For the Layer Node Node embedding, is the hidden representation of the node, For the The learning parameters of the layer attention head, is a feed-forward neural network, is the activation function, the operator Represents a splicing operation, express Taken from The adjacent nodes of , Indicates Layer Node The total number of attention heads is .

[0050] The updating process of each node embedding based on the attention mechanism is expressed as follows: ; in, , Indicates function block In the , The feature vector of the layer's functional attributes, Indicates function block In the The feature vector of the layer's functional attributes, Indicates Layer attention learning parameters, Represents a node pair The attention weight, Represents a nonlinear transformation.

[0051] This embodiment integrates the nodes Its adjacent nodes Information features based on nonlinear transformation Update , , Representation Node , The number of adjacent nodes.

[0052] Through the above steps, for the node Generate a low-dimensional graph embedding vector and update , which contains both the functional attributes of the node itself and its contextual relationship in the graph.

[0053] Step 3: Introduce a large language model to learn the calling relationship between the function blocks of benign samples. This embodiment uses a large language model to analyze benign samples, understand their execution logic, learn and store the calling relationship of benign samples. On this basis, the pre-trained large language model predicts the function block calling relationship of unlabeled samples, and locks the abnormal node pairs after completing the edge assignment, which is convenient for subsequent targeted adversarial training. The specific process is as follows: Figure 3 shown.

[0054] Step 3 specifically includes the following steps: Step 3.1: Submit the multidimensional data obtained in the preprocessing stage to the large language model. LLM learns the logical relationships in benign samples and deeply understands the semantic relevance between different functional blocks and their calling patterns. In this embodiment, the generated graph embedding vector, the original opcode sequence and the API call sequence are combined to construct the input sequence as follows: ; in, is the original opcode sequence, is the original sequence of API calls.

[0055] Step 3.2: Learn the call relationship between benign samples. Use a labeled database to train LLM, and the large language model is pre-trained using masked language modeling (MLM) and sequence generation tasks. In masked language modeling, some opcodes, API names, or elements in graph embeddings are randomly masked to let the model predict the masked content. This embodiment is based on the bidirectional encoder representations from transformers (BERT) model to complete the training, and its cross entropy loss function is for: ; in, represents the parameters of the transformer encoder and output layer in BERT, represents the set of masked tokens in the training phase, Indicates In each self-attention layer in BERT, an input masked token is embedded by updating the weights of the embeddings of other concatenated tokens, and the embedding of each token captures context-sensitive functional semantic information that varies with position and context.

[0056] Specifically, we input "graph embedding [0.12, 0.34, [MASK], 0.56], opcode sequence {mov; cmp; [MASK]}, API call sequence {CreateFileW; [MASK]}" to predict graph embedding, opcode, and API call information. We use accuracy and perplexity to evaluate the model's ability to understand the input information, and improve the model's robustness through data augmentation (randomly shuffling the opcode order, replacing API calls with synonyms, etc.).

[0057] Step 3.3: Iteratively use the pre-trained model to update the call probability until the call relationship probability matrix of the function blocks in the entire benign binary file sample library converges. Indicates the probability of calling between function blocks, and the construction prompt word is ,in, It is a function block Each iteration adjusts the probability value according to the model's prediction and the actual call data, gradually approaching the actual call distribution.

[0058] Specifically, the graph embedding vector, opcode sequence, and API call sequence are concatenated into text form according to certain rules, and the corresponding task description is added. For example, "The following is the relevant information of the function block, please understand its function and calling relationship, graph embedding [0.12, 0.34, ..., 0.56], opcode sequence {mov; cmp; je}, API call sequence {CreateFileW; ReadFile}, learn its calling probability with other adjacent function blocks".

[0059] Each subsequent iteration will adjust the probability value based on the model's prediction and actual call data, gradually approaching the actual call probability distribution.

[0060] The update formula of the call probability matrix is: ; in, , Indicates , The first iteration Function Blocks To Function Blocks The call probability matrix, represents the iterative learning rate, Indicates Function Blocks To Function Blocks The actual number of calls, the iteration termination condition is that the maximum change of the calling probability matrix is ​​less than the change threshold ,Right now .

[0061] During the model training process, the call probability loss function between nodes is recorded as , the formula is as follows: ; in, is the parameter vector used in model training, Indicates that a given node in the training sample and parameter vector In the case of calling node The probability of and Respectively represent the number of samples and the number of nodes of a single sample. On this basis, optimize the model parameters as follows: ; in, represents the model learning rate, Represents the loss function with respect to the parameter The gradient of express The elements in is a regularization hyperparameter, introducing the regularization term Prevent model overfitting and improve model generalization ability.

[0062] Step 3.4: Store the call probability. The call probability of the benign sample will serve as the basis for subsequent analysis and decision-making, providing the possibility of calls between various function blocks in the program. Select an efficient data structure to store the final stable call probability matrix , this embodiment uses a balanced binary tree AVL tree for storage. Among them, the query operation is , the update operation is , which aims to efficiently find and adjust the calling probability matrix between function blocks.

[0063] Step 4: Multi-objective planning to find the optimal operation sequence. Considering that the control flow transformation of malware is a discrete and high-dimensional space, this embodiment is based on the reinforcement learning (RL) agent to efficiently search for the optimal adversarial operation sequence. Through continuous trial and adjustment, it aims to find an operation sequence that retains the core functions of the malware and effectively evades security detection at the lowest cost, providing key theoretical support for subsequent malware reconstruction.

[0064] Step 4 specifically includes the following steps: Step 4.1: Evaluate the nodes in the current sample With benign sample nodes The correlation between : ; in, and Represent the current software and benign sample set respectively The feature set of functional attributes, Indicates the number of features. The numerator indicates the number of similar features between benign samples and malicious samples, and the denominator indicates the number of benign sample features. Finally, the comprehensive similarity between the current software and the benign sample data set is obtained. . Design evaluation threshold ,Similarity lower than the threshold indicates high abnormality and adversarial confusion is required.

[0065] Step 4.2: Use the benign sample data in the memory buffer to train the Q learning network and determine the optimal obfuscation operation. Get in ascending order Abnormal nodes are submitted to the RL agent to complete the obfuscation work.

[0066] Monte Carlo Tree Search (MCTS) allows finding the optimal sequence to successfully evade the target malware detection system through simulation in a large discrete space without prior information about the target system, that is, without having a deep understanding of the internal working mechanism of the target detection system.

[0067] Specifically, MCTS is selected as the RL agent, the updated CFG graph structure is input, and the transformation sequence is output after multi-objective planning. , aiming to achieve the effect of evading detection with minimal obfuscation cost: .

[0068] This embodiment studies the black box detection model in general, where: Represents the detection result of the alternative detector, and the sequence limit length is Each time a function block is selected, Iterate the sub-optimization algorithm to find the function block with the highest reward value and perform the operation. Then update the selected basic block and the corresponding operation to sequence. Each time, the Monte Carlo tree is used to calculate the search sequence, and the reward function of the adversarial sample is obtained to assign a value to each participating functional block.

[0069] The updated control flow graph of the malicious sample is represented as the initial state , for each time step, the RL agent chooses an action The actions include Nops semantic injection, call transfer and node hiding of the current node and neighboring nodes. Among them, the Nops instruction is a special instruction that does not perform any operation and is often used for code obfuscation. In this embodiment, the RL agent calculates the decision reward value ,in, Shows the current state of the graph. Indicates the operation performed on the selected function block. The probability of escaping the model is calculated as , Represents the target label, which is 1 if the probability increases, otherwise 0.

[0070] The reward function for the reinforcement learning agent is: ; in, Indicates The agent is based on the state Select Action The reward value returned by the environment, , Indicates , The probability of escaping detection in time steps is calculated, and the optimal operation sequence is searched by maximizing the cumulative reward value through the Q-function reinforcement learning algorithm.

[0071] Based on the reward function, the Q function is defined as: Take action The expected return that can be obtained is is the expected function, taking the constant is the discount factor, and the formula is as follows: .

[0072] Use the Q-function reinforcement learning (Q-learning) algorithm to learn the optimal strategy and iteratively update the Q function. Among them, the complexity of selecting instruction injection is , is the number of semantic Nops applied in this embodiment. Once the injected instructions reach the constraint or escape successfully, the reinforcement learning process will stop and feedback the current optimal operation sequence.

[0073] In MCTS, each node Represents a state , the obfuscation strategy is the action In this embodiment, the obfuscation strategy includes semantic injection, call transfer and node hiding. The probability of calculating the evasion model is , represents the target label, which is 1 if the probability is increased, otherwise 0. The reward value is: .

[0074] This embodiment obtains abnormal node ranking information based on node similarity and uses it as prior knowledge to initialize the node priority of the MCTS tree. In the MCTS selection phase, the upper confidence bound calculation formula (UCB) is optimized, considering the number of node visits, reward value and abnormal ranking to balance exploration and utilization. The specific formula is: ; in, Representation Node Score for the selection phase, first item represents the average reward value, Representation Node Reward value, Representation Node The number of visits; the second item For exploration items, MCTS is encouraged to select nodes that are less visited. Representation Node The parent node of The number of visits, constant term Controls the exploration intensity, adjusted in the interval [0.5, 2.0] by grid search, with a default value of ; Item 3 A reference item newly added to this embodiment, Indicates the abnormal value of the node, constant term Control the priority of abnormal nodes, dynamically scale according to similarity distribution, the default value is Secondly, based on The nodes with the highest scores are selected in turn for expansion, simulation, and backtracking to quickly and efficiently identify and handle abnormal nodes.

[0075] Step 4.3: In this embodiment, one of the actions is to call transfer and node hiding. First, query the nodes in the neighborhood that are highly similar to the abnormal node. If there are similar nodes, hide and transfer operations are performed. The model quantifies the similarity between nodes in the same sample based on the k-Nearest Neighbors (KNN) algorithm, and determines the node similarity by calculating the distance between feature vectors: ; in, , Represents a node pair Opcode similarity, API call similarity, , Represents a node pair , node pair The distance between the Opcode operation code vectors, , Represents a node pair , node pair The distance between the API call vectors, Indicates the number of nodes.

[0076] According to the Euclidean distance of node features, select The node closest to it in the neighborhood , lock the nodes with abnormal call relationships for obfuscation. For specific operations, see step 5.2.

[0077] Step 4.4: If there is no similar node in the neighborhood of the abnormal node, select the action "Inject Nops instructions commonly seen in benign sample sets". Instructions include but are not limited to add / sub, push / pop, xor, mov, test, and, cmp, or, etc. The injection location is the function block header.

[0078] Specifically, the Nops instruction is injected into the head of the current node, and after the Nops instruction injection is completed, The instruction completes the logical jump, see step 5.3 for details, and is required to meet the executable constraints.

[0079] Step 5: Implement adversarial software reconstruction based on executable file constraints This embodiment implements fine-grained obfuscation for control flow and reconstructs executable adversarial software on this basis. During the reconstruction process, nodes and edges are fine-tuned and optimized. Instructions implement function jumps and calls. The model needs to retain the core functions of the original malware while effectively evading existing security protections.

[0080] Step 5 specifically includes the following steps: Step 5.1: Clarify the node selection strategy and confusion action set. According to the adjacency matrix , based on node functional attributes Find the node with the highest similarity, that is The node in the adjacent range that is closest to its Euclidean feature . judge and Does the similarity exceed the threshold? If the threshold is exceeded, the action selected is to call transfer and node hiding, otherwise the semantically irrelevant instruction Nops is injected into the node.

[0081] Step 5.2: Call transfer and node hiding. and The one with the smallest probability of being called is , the one with the highest probability of being called is recorded as , this embodiment is Implement call transfer and hiding.

[0082] Specifically, the call The parent node of , inserted before its final jump / call instruction Instructions pointing to , complete the abnormal node The called relationship is transferred. Complete the polymorphic operation of input and output and add the call relationship label. The input and output parts complete the branch jump according to the label: if the call The parent node is , then in Insert before the last instruction of the block Instruction, jump to the hidden node Child nodes of Based on the above, complete the nodes within the neighborhood , that is, to realize the node "deletion" and edge "transfer" operations.

[0083] Specifically, if the functional attributes of node 002 and node 003 are similar, and the probability of node 003 being called is small, then , ,The specific call transfer is shown in Table 2.

[0084] Table 2: Instructions for calling transfer operations.

[0085] .

[0086] Among them, When a node completes a polymorphic input and output operation, it must retain the corresponding Call relationship label, that is The parent and child node numbers of the nodes. Taking the call relationship "001→003→004" as an example, node 003 is hidden due to an exception, and its function is completed by the polymorphic node 002. At this time, the corresponding polymorphic label is 002 (1-4), where 1 represents the parent node 001, 4 represents the child node 004, and so on. After clarifying the node polymorphic label, you need to Node head and tail injection correspondence Instruction, where 002 (1-4)_last represents the last instruction of the 002 node pair (1-4) label after the polymorphism, completing the executable. The specific visualization process is as follows Figure 4 shown.

[0087] Step 5.3: Inject semantically irrelevant instructions Nops into the node. First, and Respectively Specifically, the size and address of the free space in the ".text" section of the current node The first address is , the length of the first instruction is , the length of the second instruction is In this embodiment, the node The second instruction is The command is updated to ,make The first instruction at address is The original second instruction Then inject Nops instructions to confuse it. Finally, add Instruction, jump to node The third instruction Make sure that the space required for all injected instructions does not exceed , in accordance with executable conditions and constraint specifications, the specific visualization process is as follows Figure 5 shown.

[0088] Step 6: Evaluate the performance of adversarial training. In the field of security research and malware attack and defense, important indicators for measuring the performance of adversarial training include attack success rate ASR and semantic retention rate SPR. The attack success rate reflects the ability of malware to bypass security detection, while the semantic retention rate measures the degree to which the core functions of the software are retained after adversarial training. This embodiment optimizes the above evaluation indicators, and after performance evaluation, feedback is given to the reinforcement learning agent to build an attack and defense dynamic game framework.

[0089] Step 6 specifically includes the following steps: Step 6.1: Calculate the attack success rate ASR. ASR is the most commonly used indicator for evaluating adversarial attacks. It is defined as the proportion of all malware that successfully bypasses the target system detection. The calculation formula is as follows: ; in, Indicates the attack success rate, Indicates the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. Represents the original malicious sample, represents adversarial examples, Represents the cardinality of the set, i.e., the number of adversarial software that conform to the executable file format constraints.

[0090] Step 6.2: Calculate the semantic retention rate SPR. The existing semantic retention rate only considers the API sequence. This embodiment combines the API sequence and the Opcode sequence to measure the semantic retention. The difference between the adversarial software and the original software is calculated based on the API call sequence, which is recorded as , the calculation formula is as follows: ; in, , Indicates a malware sample , adversarial software samples API sequence features, , Indicates a malware sample , adversarial software samples The characteristic length of the API sequence, express and difference.

[0091] Similarly, the difference based on the Opcode sequence is recorded as , the calculation formula is as follows: ; in, , Indicates a malware sample , adversarial software samples Opcode sequence features, , Indicates a malware sample , adversarial software samples The characteristic length of the Opcode sequence, express and difference.

[0092] Comprehensive consideration, use weights API sequence differences between fused adversarial malware and original malware And Opcode sequence difference , get the sample difference .

[0093] On this basis, the semantic similarity is calculated The formula is as follows: ; in, Represents the evaluation threshold determined by the average difference of the samples.

[0094] Considering that some random calls to the API in the sandbox may lead to the same software In this embodiment, the same sandbox is used to analyze all original malware samples twice, and the average difference of all samples is calculated to determine Finally, the semantic preservation rate (SPR) is defined as the proportion of adversarial malware that retains the original semantics among adversarial malware that successfully bypasses detection, and the calculation formula is as follows: ; in, represents the semantic retention rate, Indicates the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. Represents the original malicious sample, represents adversarial examples, express and The semantic similarity of Represents the cardinality of the set, i.e., the number of adversarial software that conform to the executable file format constraints.

[0095] Step 6.3: Feed the actual evaluation results back to the reinforcement learning RL agent, compare and analyze the predicted results with the actual results to update the alternative detector, improve the reward function in the reinforcement learning agent, continuously optimize the simulation process, build a dynamic game framework, and improve the performance of the adversarial training system.

[0096] Specifically, the actual evaluation results are fed back to the Monte Carlo tree MCTS, the predicted results are compared and analyzed with the actual results to update the replacement detector, the reward function in the reinforcement learning agent is improved, the simulation process is continuously optimized and trained, and the following is constructed: Figure 6 The dynamic game framework shown in Figure 1 improves the performance of the adversarial training system.

[0097] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction, characterized in that: include: Use static analysis tools to parse executable samples, extract control flow graphs and optimize them into local function block structures, and generate functional attribute characteristics of nodes; Aggregate functional attribute features and topological relationships based on graph attention mechanism to generate low-dimensional graph embedding; The low-dimensional graph embedding, opcode sequence and API call sequence are input into the large language model to learn the function block call probability distribution of benign samples and lock the abnormal node pairs; Based on reinforcement learning agent, the optimal operation sequence including call transfer, node hiding and semantic irrelevant instruction injection is searched with the attack success rate and semantic preservation rate as the target; Adjust the control flow graph according to the optimal operation sequence and reconstruct the adversarial software that complies with the executable file format constraints; By dynamically evaluating the attack success rate and semantic retention rate, feedback is provided to optimize the reinforcement learning agent and form a dynamic attack and defense game framework.

2. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: The functional attribute characteristics of the generated node include: Extract the operation code sequence of the function block and generate a numerical operation code feature vector by counting the frequency; Parse the API call sequence and generate semantic-level API feature vectors through a pre-trained language model; The operation code feature vector and the application programming interface feature vector are concatenated to form the functional attribute feature of the node.

3. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: Generating low-dimensional graph embeddings includes: dynamically allocating weights of neighborhood nodes through a multi-head attention mechanism, combining multi-hop neighborhood aggregation and nonlinear transformation to update node features, and generating low-dimensional graph embeddings that include functional attribute features and topological relationships.

4. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: Learning the function block call probability distribution of benign samples includes: training a large language model through masked language modeling and sequence generation tasks, iteratively updating the call probability matrix between function blocks until convergence, and storing the call probability matrix in a balanced binary tree for query and update.

5. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 4 is characterized in that: The update formula of the call probability matrix is: ； in, , Indicates , The first iteration Function Blocks To Function Blocks The call probability matrix, represents the iterative learning rate, Indicates Function Blocks To Function Blocks The actual number of calls, the iteration termination condition is that the maximum change of the calling probability matrix is ​​less than the change threshold .

6. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: The reward function for the reinforcement learning agent is: ； in, Indicates The agent is based on the state Select Action The reward value returned by the environment, , Indicates , The probability of escaping detection in time steps is calculated, and the optimal operation sequence is searched by maximizing the cumulative reward value through the Q-function reinforcement learning algorithm.

7. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: The call transfer includes: inserting a jump instruction to point to a high-probability node in the parent node instruction of the call exception node, and adding an instruction to jump to the sub-address of the hidden node at the end of the high-probability node to ensure the logical coherence of the control flow.

8. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1 is characterized in that: Semantically irrelevant instruction injection includes: injecting semantically irrelevant instructions into the header of the function block, and redirecting the execution flow to the original instruction sequence through jump instructions, ensuring that the space occupied by the injected instructions complies with the executable file format constraints.

9. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1, characterized in that: The calculation formula for the attack success rate is: ； in, Indicates the attack success rate, Indicates the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. Represents the original malicious sample, represents adversarial examples, Represents the cardinality of the set, i.e., the number of adversarial software that conform to the executable file format constraints.

10. The adversarial training method for fine-grained flow obfuscation based on semantic probability reconstruction according to claim 1, characterized in that: The semantic retention rate is calculated by fusing the normalized difference between the opcode sequence and the API call sequence, and the calculation formula is: ； in, represents the semantic retention rate, Indicates the detection result of the malware detector on the sample. A detection result of 0 indicates a benign sample, and a detection result of 1 indicates a malicious sample. Represents the original malicious sample, represents adversarial examples, express and The semantic similarity of Represents the cardinality of the set, i.e., the number of adversarial software that conform to the executable file format constraints.

Images