Adversarial Sample Generation Method Based on APT Group Malicious Code

Abstract

This invention discloses a method for generating adversarial samples based on APT (Advanced Persistent Threat) malware. First, a sequence pattern mining algorithm is used to extract a set of benign subsequences from benign code. Next, a directed graph is constructed for each benign subsequence to obtain a set of perturbation actions. Then, the original malware sequence is input into a target detection model for detection, identifying the most vulnerable positions in the original malware sequence. Subsequently, benign subsequences are selected from the perturbation action set and inserted into the most vulnerable positions. A deep Q-network algorithm is used to find the optimal inserted subsequence to obtain the adversarial sequence. Finally, the adversarial sequence is applied to actual malware to obtain the adversarial sample. This application, by constructing a perturbation action set and optimizing the insertion using a deep Q-network algorithm, generates adversarial samples that can successfully bypass existing detection models while retaining the original functionality of the malware, thus enhancing the stealth of the attack.

Description

Technical Field

[0001] This invention belongs to the technical field of malware detection, specifically relating to a method for generating adversarial samples based on APT group malicious code. Background Technology

[0002] In recent years, the cybersecurity field has faced increasingly severe challenges, among which APT (Advanced Persistent Threat) groups pose a serious threat to enterprises, government agencies, and other critical organizations through highly sophisticated and customized malicious code. APT attacks are typically characterized by long-term latency, strong stealth, and targeted attacks, making traditional security defenses often ineffective against these advanced threats.

[0003] Existing malware detection methods primarily rely on static signature analysis and signature-based detection techniques. Static analysis methods identify malware by examining file signatures and structural information. However, attackers can easily evade static analysis through code obfuscation, packing, and encryption, making these methods ineffective against modern, advanced malware. While dynamic analysis can capture the behavioral characteristics of malware at runtime, it also faces the challenge of evasion techniques. For example, malware can use environment detection and dynamic loading techniques to identify virtual environments and alter its behavior, thereby evading detection.

[0004] Existing detection systems often rely on known malware signature databases, which contain signatures or behavioral patterns of known malware. However, APT groups frequently adjust their attack methods dynamically based on the target environment, making traditional signature-based detection methods ineffective against new and unknown threats. In recent years, with the widespread application of machine learning and deep learning technologies, researchers have attempted to leverage these techniques to improve the efficiency and accuracy of malware detection. However, existing machine learning-based detection methods typically focus only on a single type of malware feature (such as static features or dynamic behavior) or tend to build a single classification model. While these methods may perform well in specific situations, their performance often falls short in real-world environments such as uneven distribution of malware family samples, insufficient sample size, and varied attack methods.

[0005] In summary, existing malware detection technologies have significant shortcomings in addressing advanced threats from APT groups. Traditional technologies struggle to adapt quickly to changes in malware and complex attack methods, lacking effective detection capabilities against new threats. Therefore, there is an urgent need to develop a new method capable of dynamically generating adversarial samples to simulate and predict potential variants of malware and attack behaviors, thereby improving the robustness and adaptability of detection systems. This is precisely the problem that this invention aims to solve. Summary of the Invention

[0006] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a method for generating adversarial samples based on APT malicious code. This method utilizes the deep Q-network algorithm to generate adversarial samples in real time, simulating and predicting the attack behavior of APT organizations, thereby enhancing the robustness and response capability of network defense systems and models.

[0007] To achieve the above objectives, the present invention provides a method for generating adversarial samples based on malicious code from APT groups, comprising the following steps:

[0008] Use a sequence pattern mining algorithm to extract a set of benign subsequences from benign code;

[0009] Based on the set of benign subsequences, construct a directed graph for each benign subsequence to obtain the set of perturbation actions;

[0010] The original malicious sequence is input into the target detection model for detection. By calculating the gradient of the loss function of the target detection model, the most vulnerable position in the original malicious sequence is identified.

[0011] Select benign subsequences from the set of perturbation actions and insert them into the most vulnerable positions. Use the deep Q-network algorithm to find the best-performing subsequences to obtain the adversarial sequence.

[0012] Adversarial sequences are applied to actual malicious code. The malicious code is compiled into an intermediate representation using the LLVM toolchain, and system calls or API calls are inserted at predetermined locations in the malware source code to obtain adversarial samples.

[0013] As a preferred technical solution, the step of using a sequence pattern mining algorithm to extract a set of benign subsequences from benign code specifically involves:

[0014] Obtain the set of benign behavior sequences from existing benign code;

[0015] Set a minimum support threshold and an empty set of benign subsequences;

[0016] Obtain the frequency of each benign behavior sequence in the benign behavior sequence set. If the frequency is greater than the minimum support threshold, add it to the benign subsequence set.

[0017] As a preferred technical solution, for a benign subsequence B containing m different benign behaviors... i ={b i1 ,b i2 ,...,b ik ,...,b im},

[0018] The benign subsequence B iConstruct a directed graph \(G\) with the benign behaviors in it as nodes and the sequential relationships between the benign behaviors as edges. i , then for the directed graph \(G\) i , the node set \(N\) i = \(B\) i , and the edge set \(E\) i = \(\{(b\) ik , \(b\) ik+1 ) | 1 ≤ \(k\) < \(m\}\), where \(b\) ik is the \(k\)-th benign behavior in the \(i\)-th benign subsequence, \(b\) ik+1 is the \((k + 1)\)-th benign behavior in the \(i\)-th benign subsequence, and \((b\) ik , \(b\) ik+1 ) is the edge between the benign behaviors \(b\) ik and \(b\) ik+1 ;

[0019] Obtain the adjacency matrix \(C\) i of the directed graph \(G\) i :

[0020]

[0021] where, \(e\) ij represents the sequential relationship between the benign behaviors \(b\) ix and \(b\) iy , that is, when the benign behavior \(b\) [[ID=…]] iy appears before the benign behavior \(b\) ix and there is an edge between them, then \(e\) ij = 1, otherwise \(e\) ij = 0; \(b\) ix is the \(x\)-th benign behavior in the \(i\)-th benign subsequence, and \(b\) iy is the \(y\)-th benign behavior in the \(i\)-th benign subsequence.

[0022] As a preferred technical solution, the method for identifying the most vulnerable position in the original malicious sequence is specifically as follows:

[0023] For a given original malicious sequence \(S=\{S_1,S_2,\cdots,S\) i ,\cdots,S\) n \}, input it into the target detection model to obtain the gradient vector of the loss function:

[0024]

[0025] where, is the gradient vector of the loss function, \(S\) i is the \(i\)-th malicious behavior in the original malicious sequence, is the gradient magnitude of the \(i\)-th malicious behavior in the original malicious sequence, and \(n\) is the number of malicious behaviors in the original malicious sequence;

[0026] The location with the largest gradient magnitude is identified based on the gradient vector and is thus the most vulnerable to attack. The selection method is as follows:

[0027]

[0028] Among them, i * The most vulnerable position Let represent the gradient magnitude of the i-th malicious behavior in the gradient vector of the loss function, and n represent the number of malicious behaviors in the original malicious sequence.

[0029] As a preferred technical solution, the obtained adversarial sequence specifically refers to:

[0030] The original malicious sequence is used as the sequence to be modified. The state of the sequence to be modified is defined as the combination of the sequence to be modified and the most vulnerable position. The action is defined as selecting a benign behavior from the benign subsequence to be inserted and inserting it into the most vulnerable position of the sequence to be modified.

[0031] The state of the sequence to be modified is input into a deep Q-network algorithm, and a new sequence is obtained by selecting actions using a greedy strategy through the Q-value update function.

[0032] Simultaneously, the new sequence is input into the object detection model for prediction, and the reward function of the Q-value function is updated based on the loss function of the object detection model;

[0033] A heuristic backtracking search algorithm is used to constrain the new sequence;

[0034] Repeatedly perform action selection, prediction, and constraint until all benign behaviors of the benign subsequence to be inserted have been inserted or the reward function reaches the set threshold, to obtain the optimal new sequence and thus the adversarial sequence.

[0035] As a preferred technical solution, the Q-value function is represented by the Bellman equation:

[0036]

[0037] Where r is the reward function, γ is the discount factor, and s t+1 The state of the sequence to be modified after taking action a, a t+1 For the sequence to be modified in state s t+1 The selected action, s t Given the current state of the sequence to be modified, a t For the sequence to be modified in the current state s t The chosen action...

[0038] As a preferred technical solution, the reward function is constructed based on the loss function of the object detection model, and is expressed as:

[0039]

[0040] Where s is the state of the sequence to be modified, a is the action chosen by the sequence to be modified in state s, and s ’ The state of the sequence to be modified after taking action 'a'. Here are the parameters of the object detection model; α is the scaling factor, used to adjust the impact of the loss function on the reward function. Let y be the loss function of the object detection model, y be the true label of the new sequence, and y = 0 indicate that the new sequence is a benign sequence.

[0041] As a preferred technical solution, the method of using a heuristic backtracking search algorithm to constrain the new sequence specifically includes:

[0042] Obtain the directed graph of the benign subsequence to be inserted, calculate the weight of the benign behavior in the benign subsequence to be inserted at the most vulnerable position and sort them;

[0043] If the new sequence is predicted to be benign by the target detection model, but the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of the benign subsequence to be inserted, then backtrack to the previous state of the new sequence and select the benign behavior with the highest weight from the successor nodes of the benign behavior in the previous state of the new sequence for insertion.

[0044] When a new sequence is classified as malicious by the target detection model, and the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of the benign subsequence to be inserted, another benign subsequence is selected from the perturbation action set as the benign subsequence to be inserted, and the action selection is restarted from the original malicious sequence.

[0045] Another aspect of the present invention provides an adversarial sample generation system based on APT group malicious code, characterized in that the system includes a benign screening module, a perturbation construction module, a location identification module, a sequence generation module, and a code modification module;

[0046] The benign screening module is used to extract a set of benign subsequences from benign code using a sequence pattern mining algorithm;

[0047] The perturbation construction module is used to construct a directed graph for each benign subsequence based on the set of benign subsequences, thereby obtaining a set of perturbation actions;

[0048] The location identification module is used to input the original malicious sequence into the target detection model for detection, and to identify the most vulnerable position in the original malicious sequence by calculating the gradient of the loss function of the target detection model.

[0049] The sequence generation module is used to select benign subsequences from the set of perturbation actions and insert them at the most vulnerable positions. The deep Q-network algorithm is used to find the best inserted subsequence to obtain the adversarial sequence.

[0050] The code modification module is used to apply adversarial sequences to actual malicious code, compile the malicious code into an intermediate representation using the LLVM toolchain, and insert system calls or API calls at predetermined locations in the malware source code to obtain adversarial samples.

[0051] In another aspect, the present invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the aforementioned adversarial sample generation method.

[0052] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0053] 1. Dynamically generate adversarial examples:

[0054] This invention utilizes a deep Q-network algorithm to dynamically generate adversarial examples, enabling the target detection model to have stronger adaptability and robustness, and effectively cope with advanced threats from APT groups.

[0055] 2. Effectively bypass detection mechanisms:

[0056] By constructing a set of perturbation actions and combining it with an optimized insertion strategy, the adversarial samples generated by this invention can successfully bypass existing detection models while retaining the original functionality of the malicious code, thus enhancing the stealth of the attack.

[0057] 3. Intelligent and efficient:

[0058] This invention achieves intelligent generation of adversarial examples through an automated code modification process, improving the efficiency of the generation process and reducing the time and resources required for manual intervention.

[0059] 4. Logical rationality and semantic validity:

[0060] The backtracking search algorithm is used to ensure that the generated adversarial examples are logically sound and semantically valid, avoiding the problems of functional loss and unreasonable code insertion that occur in traditional methods. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is an overall flowchart of the adversarial sample generation method based on APT group malicious code in an embodiment of the present invention.

[0063] Figure 2This is a schematic diagram of the adversarial sample generation system based on APT group malicious code in an embodiment of the present invention.

[0064] Figure 3 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Detailed Implementation

[0065] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0066] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0067] like Figure 1 As shown, this embodiment provides a method for generating adversarial samples based on APT group malicious code, including the following steps:

[0068] S1. Use a sequence pattern mining algorithm to extract a set of benign subsequences from benign code.

[0069] In the adversarial example generation process, data mining is a crucial step, requiring the extraction of representative features from existing benign code to construct subsequent perturbation sets. In this step, sequence pattern mining algorithms are used to extract representative patterns from the benign code, resulting in a set of benign subsequences, specifically:

[0070] S101. Obtain the set of benign behavior sequences from existing benign code;

[0071] S102. Set the minimum support threshold and the set of empty benign subsequences;

[0072] S103. Obtain the frequency of each benign behavior sequence in the benign behavior sequence set. If it is greater than the minimum support threshold, add it to the benign subsequence set.

[0073] The benign subsequences obtained through this mining method will better reflect the behavioral patterns of benign code, thus making them more representative.

[0074] S2. Construct a directed graph for each benign subsequence based on the set of benign subsequences to obtain a set of perturbation actions.

[0075] Construct a set of perturbation actions from the extracted set of benign subsequences, and use a directed graph G to represent the order relationship of each benign subsequence. The set of perturbation actions implies the range of possible perturbations that can be introduced by selectively inserting various benign behavior segments into the original malicious behavior sequence, thereby enabling the perturbed sequence to confuse the target detection model.

[0076] Further, for a benign subsequence B i ={b i1 , b i2 ,..., b ik ,..., b im} containing m different benign behaviors, use the benign behaviors in the benign subsequence B i as nodes and the order relationship between the benign behaviors as edges to construct a directed graph G i . Then, the node set N i of the directed graph G i = B i , and the edge set E i = {(b ik , b ik+1 ) | 1 ≤ k < m}, where b ik is the k-th benign behavior in the i-th benign subsequence, b ik+1 is the (k + 1)-th benign behavior in the i-th benign subsequence, and (b ik , b ik+1 ) is the edge between the benign behaviors b ik and b ik+1 ; then obtain the adjacency matrix C i of the directed graph G of the i-th benign subsequence: i :

[0077]

[0078] where, e ij represents the order relationship between the benign behaviors b<00000​​​​​​​​​​​​​​

[0079] By traversing a directed graph starting from different initial nodes and selecting successor nodes based on the graph's structure, different sequences of behaviors can be generated. This allows for a wider range of sequence variations while satisfying the sequence relationships between benign behaviors. Inserting these benign sequence fragments into malicious sequences helps improve the complexity and semantic effectiveness of adversarial examples. For example, consider the following directed graph containing five benign behaviors: b1, b2, b3, b4, and b5, which has six benign behavior order relationships: b1→b2, b1→b3, b2→b4, b2→b5, b3→b4, b4→b5. By traversing this directed graph, the following possible sequences of benign behaviors can be generated: starting from b1, traversing the graph yields the sequences (b1, b2), (b1, b2, b4), (b1, b2, b4, b5), (b1, b3), (b1, b3, b4), (b1, b3, b4, b5). Similarly, starting from b2, traversing the graph yields the sequences (b2, b4) and (b2, b4, b5), which demonstrates the flexibility and ability of directed graphs to represent behavioral patterns and generate various benign behavioral sequences.

[0080] S3. Input the original malicious sequence into the target detection model for detection. By calculating the gradient of the loss function of the target detection model, the most vulnerable position in the original malicious sequence is identified.

[0081] Furthermore, the most effective modification locations can be found by utilizing the gradient of the loss function of the object detection model relative to its position in the sequence, rather than randomly modifying the behavior sequence. Locations with higher gradient values ​​have a more significant impact on the loss function, indicating that they are more easily modified. This method ensures that modifications made to malicious behavior sequences have the greatest impact on their classification results. Specifically:

[0082] Given a given original malicious sequence S = {S1, S2, ..., S...} i ,...,S n The gradient vector of the loss function is obtained by inputting it into the object detection model.

[0083]

[0084] in, Let S be the gradient vector of the loss function. i For the i-th malicious action in the original malicious sequence, Let represent the gradient magnitude of the i-th malicious behavior in the original malicious sequence, and n represent the number of malicious behaviors in the original malicious sequence.

[0085] The location with the largest gradient magnitude is identified based on the gradient vector and is thus the most vulnerable to attack. The selection method is as follows:

[0086]

[0087] Among them, i * The most vulnerable position Let represent the gradient magnitude of the i-th malicious behavior in the gradient vector of the loss function, and n represent the number of malicious behaviors in the original malicious sequence.

[0088] S4. Select benign subsequences from the set of perturbation actions and insert them at the most vulnerable positions. Use the deep Q-network algorithm to find the best inserted subsequence to obtain the adversarial sequence.

[0089] Furthermore, this step learns the anomalous behavioral characteristics of the sequence to determine the modification strategy at the most vulnerable position in the sequence, ultimately generating adversarial examples. This application utilizes a Deep Q-Network (DQN) algorithm to guide the modification process, aiming to iteratively modify the original malicious sequence until the target detection model classifies it as benign. Specifically:

[0090] S401. Take the original malicious sequence as the sequence to be modified, and define the state of the sequence to be modified as a combination of a benign subsequence to be inserted and the most vulnerable position of the sequence to be modified; define the action as randomly selecting a benign subsequence from the perturbation action set and inserting it into the most vulnerable position of the sequence to be modified.

[0091] More specifically, the state is represented by a sequence of actions, that is, the sequence of actions after inserting the benign subsequence to be inserted at the most vulnerable position in the sequence to be modified; each sequence is encoded using one-hot encoding, where each value corresponds to a specific action. The state vector s∈S* is a sequence of actions of length n, where each element of the vector s corresponds to a specific action, represented by a number s. i , and s i ∈R, where 1≤i≤n; the complete state S* can be represented as: S*={s|s=(s1,s2,...,s n ),s i ∈R, 1≤i≤n}.

[0092] The state is jointly handled by the object detection model and the DQN algorithm. The object detection model simulates black-box detection to obtain detection results by submitting specific inputs. Since most current object detection models are black-box models, the adversarial sequences generated in this application are based on the object detection model to perform adversarial attacks. The DQN model aims to modify malicious sequences to mislead the object detection model.

[0093] Actions represent possible modifications that can be applied to the original malicious sequence. To maintain the original functionality of the program, the perturbation process is restricted to inserting only benign behaviors into the malicious sequence while preserving its original malicious function. A = {a1, a2, ..., a...} n} represents the benign subsequence B = {b1, b2, ..., b} corresponding to the operation. m The insertion action is performed at the most vulnerable position in the original malicious sequence. For each action a... i ∈A, can be represented as: Insert(S,b) i ,p)→S ’ =[s1,...,s p-1 ,b i ,s p ,...,s k ];

[0094] Where S is the original malicious sequence with k malicious behaviors, S ’ For the new sequence, b i For benign insertion behavior, p is the position of benign insertion behavior, and b is the position of benign insertion behavior. i After insertion position p, the new sequence S ’ It now contains k+1 actions.

[0095] S402. Input the state of the sequence to be modified into the deep Q-network algorithm, and use the greedy strategy to select actions through the Q-value update function to obtain the new sequence.

[0096] The DQN algorithm uses reinforcement learning to optimize the modification effect. Each step of the modification process can be described as s t+1 =M(s) t ,a t ), where M is the value of a t The modification action is applied to the sequence state s to be modified. t The function is then used, and the Bellman equation is used to evaluate the Q value for taking a specific action in a specific state:

[0097]

[0098] Where r is the reward function, γ is the discount factor, and s t+1 The state of the sequence to be modified after taking action a, a t+1 For the sequence to be modified in state s t+1 The selected action, s t Given the current state of the sequence to be modified, a t For the sequence to be modified in the current state s t The chosen action. To select the most effective attack action at each step, the DQN algorithm uses the Q-value, which represents the expected long-term reward of a particular action in the current state.

[0099] S403. Simultaneously, the new sequence is input into the target detection model for prediction, and the reward function of the Q-value function is updated based on the loss function of the target detection model.

[0100] During training, DQN optimizes its modification strategy by observing the detection results of the object detection model on the modified sequence and receives a corresponding reward. This invention utilizes a reward function to guide the DQN algorithm to effectively modify the original malicious sequence that affects its classification results. The reward function is constructed based on the loss function of the object detection model and is expressed as:

[0101]

[0102] Where s is the state of the sequence to be modified, a is the action chosen by the sequence to be modified in state s, and s ’ The state of the sequence to be modified after taking action 'a'. Here are the parameters of the object detection model; α is the scaling factor, used to adjust the impact of the loss function on the reward function. Let be the loss function of the object detection model, y be the true label of the new sequence, and y = 0 indicate that the new sequence is a benign sequence (i.e., normal class). By utilizing this reward function, the DQN algorithm can be guided to minimize the loss of the normal class (label 0) proxy model, thereby effectively modifying the original malicious sequence to make it closer to a benign sequence.

[0103] In this embodiment, the prediction function of the target detection model is assumed to be: Where s represents the current state, The parameters of the corresponding object detection model; since the object detection model is a binary classifier, the loss function L... C This can be represented by the classification cross-entropy loss:

[0104]

[0105] Where y is the true label of the new sequence. This represents the predicted probability that the new sequence is predicted to be of class i.

[0106] S404. Use a heuristic backtracking search algorithm to constrain the new sequence.

[0107] More specifically, by training the DQN algorithm to learn the optimal strategy for insertion behavior, the ability of the DQN algorithm to predict the optimal insertion behavior is enhanced, and the action-value function Q(s,a;θ) continuously approximates the optimal action-value function Q. * (s,a). When using the DQN algorithm to predict the available actions, select the action that has the greatest impact on the classifier's decision at each step:

[0108]

[0109] However, the DQN algorithm may fail to account for dependencies between inserted actions when selecting the sequence of actions to be inserted. For example, suppose there is a set of actions B = {b1, b2, b3}, and their dependencies are defined by rule R. According to R, b1 must be followed by b2, and b2 must be followed by b3 to maintain the dependency constraints and thus function. However, the algorithm calculates the action estimate function Q(s) t ,a;θ t This is equivalent to solving the following optimization problem for each behavior:

[0110]

[0111] Such independent behavioral choices may lead to sequences violating the dependency constraints specified by rule R. The DQN algorithm might choose to insert b1 and b3 into the original sequence without inserting b2, resulting in an infeasible sequence: infeasible sequence = {...,b1,...,b3,...}. To address this, this application employs a heuristic backtracking search algorithm, which obtains the optimal solution by systematically trying and discarding non-optimal solutions, specifically:

[0112] Obtain the directed graph of the benign subsequence to be inserted, calculate the weight of the benign behavior in the benign subsequence to be inserted at the most vulnerable position and sort them;

[0113] If the new sequence is predicted to be benign by the target detection model, but the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of the benign subsequence to be inserted, then backtrack to the previous state of the new sequence and select the benign behavior with the highest weight from the successor nodes of the benign behavior in the previous state of the new sequence for insertion.

[0114] When a new sequence is classified as malicious by the object detection model, and the benign behavior inserted into the new sequence does not satisfy its order relationship in the directed graph of the benign subsequence to be inserted, another benign subsequence is selected from the perturbation action set as the benign subsequence to be inserted. The algorithm then backtracks to the original malicious sequence and restarts the action selection heuristic backtracking search algorithm, employing a continuous process to attempt to modify the malicious sequence while adhering to all behavioral dependency constraints. Based on the perturbation action set represented by the directed graph, when traversing various modification paths, if the generated adversarial example cannot successfully evade the classifier, it returns to an earlier state and explores alternative options. Through this iterative trial-and-error method, adversarial examples that conform to all behavioral dependency constraints can eventually be generated.

[0115] During the iterations of the DQN algorithm's main loop, the weights of the positions inserted into the sequence are reordered, and subsequent nodes are obtained by traversing all nodes in the directed graph. If the inserted behavior conforms to the dependencies defined by the directed graph, the model is allowed to predict the modified malicious behavior sequence. If the new sequence is classified as benign, the algorithm terminates successfully; if the new sequence is classified as malicious, the search modifies the next directed graph, allowing the insertion of other behaviors into the sequence. This process embodies the algorithm's backtracking characteristic. If a malicious behavior sequence cannot be modified to a benign classification, the algorithm attempts to recover its decision and explore alternative sequences. The heuristic backtracking search algorithm prioritizes promising paths and the adaptability of backtracking, reversing ineffective decisions, aiming to discover a set of behaviors that satisfy dependency constraints and obtain the desired benign classification.

[0116] S405. Repeatedly execute action selection, prediction and constraint until all benign behaviors of the benign subsequence to be inserted have been inserted or the reward function reaches the set threshold, to obtain the optimal new sequence and thus obtain the adversarial sequence.

[0117] Step S4 uses the DQN algorithm to select the optimal insertion action based on the original malicious sequence and the set of perturbation actions. This algorithm utilizes a deep neural network as a DQN agent to estimate the Q-function, which outputs the Q-values ​​of different actions, guiding the agent to choose the action that maximizes the expected reward. In each iteration, the agent model selects an action to execute according to a greedy (ε-greedy) policy, observes the reward and the next state. Then, the agent learns to maximize the expected cumulative reward, guiding its action choices towards modifying the sequence that produces the highest reward. Simultaneously, DQN has two key mechanisms: experience replay and a target network. Experience replay allows learning from past experiences, enhancing stability by breaking down correlations in the data. The target network provides a stable objective for Q-function updates; each sequence is iteratively modified until it is considered benign or the maximum number of modification attempts has been reached.

[0118] S5. Apply the adversarial sequence to actual malicious code, compile the malicious code into an intermediate representation using the LLVM toolchain, and insert system calls or API calls at predetermined locations in the malware source code to obtain adversarial samples.

[0119] The main task of this step is to apply the perturbations determined in the preceding steps to the actual malicious code to generate an adversarial sample capable of evading detection in a real-world environment. First, the malicious code is compiled into an LLVM intermediate representation (IR) file with debugging information; this format facilitates subsequent code analysis and modification. After generating the LLVM IR file, the strace tool is used to obtain the system call sequence of the malicious code at runtime, and the addr2line tool is used to map these system calls to specific source code locations. Next, predetermined system calls or API calls are inserted into these identified code locations to effectively perturb the behavioral sequence. Through this automated modification, not only is the generated adversarial sample logically sound, but it also ensures that it retains its original malicious functionality and can successfully execute on the target system.

[0120] Finally, the modified IR file was recompiled to generate an executable file. This process ensures that the malicious code's perturbations are correctly applied and that the system call sequence generated during actual runtime is consistent with expectations. This verifies the effectiveness of these modifications, i.e., whether the generated adversarial sample can evade existing detection models while retaining its original malicious functionality. This step makes it possible to move from theoretical design to practical application of adversarial samples, enhancing the defense against APT group malicious code.

[0121] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0122] Based on the same idea as the adversarial sample generation method based on APT organization malicious code in the above embodiments, the present invention also provides an adversarial sample generation system based on APT organization malicious code. This system can be used to execute the above-described adversarial sample generation method based on APT organization malicious code. For ease of explanation, the structural diagram of the embodiment of the adversarial sample generation system based on APT organization malicious code only shows the parts related to the embodiments of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0123] like Figure 2 As shown, another embodiment of the present invention provides an adversarial sample generation system based on APT group malicious code, including a benign screening module, a perturbation construction module, a location identification module, a sequence generation module, and a code modification module;

[0124] Among them, the benign screening module is used to extract a set of benign subsequences from benign code using a sequence pattern mining algorithm;

[0125] The perturbation construction module is used to construct a directed graph for each benign subsequence based on the set of benign subsequences, thus obtaining a set of perturbation actions;

[0126] The location identification module is used to input the original malicious sequence into the target detection model for detection. By calculating the gradient of the loss function of the target detection model, it identifies the most vulnerable position in the original malicious sequence.

[0127] The sequence generation module is used to select benign subsequences from the set of perturbation actions and insert them at the most vulnerable positions. The deep Q-network algorithm is used to find the best inserted subsequence to obtain the adversarial sequence.

[0128] The code modification module is used to apply adversarial sequences to actual malicious code. It compiles the malicious code into an intermediate representation using the LLVM toolchain and inserts system calls or API calls at predetermined locations in the malware source code to obtain adversarial samples.

[0129] It should be noted that the adversarial sample generation system based on APT organization malicious code of the present invention corresponds one-to-one with the adversarial sample generation method based on APT organization malicious code of the present invention. The technical features and beneficial effects described in the embodiments of the adversarial sample generation method based on APT organization malicious code described above are applicable to the embodiments of the adversarial sample generation system based on APT organization malicious code. For details, please refer to the description in the embodiments of the method of the present invention, which will not be repeated here.

[0130] Furthermore, in the above embodiments of the adversarial sample generation system based on APT organization malicious code, the logical division of each program module is only an example. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the sake of corresponding hardware configuration requirements or software implementation convenience. That is, the internal structure of the adversarial sample generation system based on APT organization malicious code is divided into different program modules to complete all or part of the functions described above.

[0131] like Figure 3 As shown, in one embodiment, a computer-readable storage medium is provided, storing a program in a memory. When the program is executed by a processor, it implements a method for generating adversarial samples based on malicious code from an APT group, specifically:

[0132] Use a sequence pattern mining algorithm to extract a set of benign subsequences from benign code;

[0133] Based on the set of benign subsequences, construct a directed graph for each benign subsequence to obtain the set of perturbation actions;

[0134] The original malicious sequence is input into the target detection model for detection. By calculating the gradient of the loss function of the target detection model, the most vulnerable position in the original malicious sequence is identified.

[0135] Select benign subsequences from the set of perturbation actions and insert them into the most vulnerable positions. Use the deep Q-network algorithm to find the best-performing subsequences to obtain the adversarial sequence.

[0136] Adversarial sequences are applied to actual malicious code. The malicious code is compiled into an intermediate representation using the LLVM toolchain, and system calls or API calls are inserted at predetermined locations in the malware source code to obtain adversarial samples.

[0137] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for generating adversarial samples based on APT group malware, characterized in that, Includes the following steps: Use a sequence pattern mining algorithm to extract a set of benign subsequences from benign code; Based on the set of benign subsequences, construct a directed graph for each benign subsequence to obtain the set of perturbation actions; The original malicious sequence is input into the target detection model for detection. By calculating the gradient of the loss function of the target detection model, the most vulnerable position in the original malicious sequence is identified. From the set of perturbation actions, select benign subsequences and insert them at the most vulnerable positions. Use a deep Q-network algorithm to find the best-performing subsequence to obtain the adversarial sequence. Specifically: The original malicious sequence is used as the sequence to be modified. The state of the sequence to be modified is defined as the combination of the sequence to be modified and the most vulnerable position. The action is defined as selecting a benign behavior from the benign subsequence to be inserted and inserting it into the most vulnerable position of the sequence to be modified. The state of the sequence to be modified is input into a deep Q-network algorithm, and a new sequence is obtained by selecting actions using a greedy strategy through the Q-value update function. Simultaneously, the new sequence is input into the object detection model for prediction, and the reward function of the Q-value function is updated based on the loss function of the object detection model; A heuristic backtracking search algorithm is used to constrain the new sequence, specifically: Obtain the directed graph of the benign subsequence to be inserted, calculate the weight of the benign behavior in the benign subsequence to be inserted at the most vulnerable position and sort them; If the new sequence is predicted to be benign by the target detection model, but the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of the benign subsequence to be inserted, then backtrack to the previous state of the new sequence and select the benign behavior with the highest weight from the successor nodes of the benign behavior in the previous state of the new sequence for insertion. When a new sequence is classified as malicious by the target detection model, and the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of the benign subsequence to be inserted, another benign subsequence is selected from the perturbation action set as the benign subsequence to be inserted, and the action selection is restarted from the original malicious sequence. Repeatedly perform action selection, prediction, and constraint until all benign behaviors of the benign subsequence to be inserted have been inserted or the reward function reaches the set threshold, to obtain the optimal new sequence and thus obtain the adversarial sequence; Adversarial sequences are applied to actual malicious code. The malicious code is compiled into an intermediate representation using the LLVM toolchain, and system calls or API calls are inserted at predetermined locations in the malware source code to obtain adversarial samples.

2. The adversarial example generation method according to claim 1, characterized in that, The sequence pattern mining algorithm is used to extract a set of benign subsequences from benign code, specifically as follows: Obtain the set of benign behavior sequences from existing benign code; Set a minimum support threshold and an empty set of benign subsequences; Obtain the frequency of each benign behavior sequence in the benign behavior sequence set. If the frequency is greater than the minimum support threshold, add it to the benign subsequence set.

3. The adversarial example generation method according to claim 1, characterized in that, For including m Benign subsequences with different benign behaviors B i = { b i1 , b i2 ,..., b ik ,..., b im }, benign subsequences B i A directed graph is constructed using benign behaviors as nodes and the sequential relationships between benign behaviors as edges. G i Then the directed graph G i Node set N i = B i edge set E i ={( b ik , b ik+1 )| 1≤ k <m },in b ik For the first i The first benign subsequence k A positive behavior, b ik+1 For the first i The first benign subsequence k +1 positive behavior, ( b ik , b ik+1 ) is a benign behavior b ik and b ik+1 The edge between; Obtaining a directed graph G i adjacency matrix C i : ， ， in, e ij Indicates benign behavior b ix and b iy The order relationship between them, that is, when benign behavior b iy Appearing in benign behavior b ix Then, if both have an edge, then... e ij =1, otherwise e ij =0; b ix For the first i The first benign subsequence x A positive behavior, b iy For the first i The first benign subsequence y A positive behavior.

4. The adversarial example generation method according to claim 1, characterized in that, The identification of the most vulnerable location in the original malicious sequence is specifically as follows: For a given original malicious sequence S ={ S 1, S 2,..., S i ,..., S n The gradient vector of the loss function is obtained by inputting it into the object detection model. ， Among them, ∇ L ( S () represents the gradient vector of the loss function. S i The first in the original malicious sequence i A malicious act, The first in the original malicious sequence i The gradient magnitude of a malicious act. n This represents the number of malicious acts in the original malicious sequence. The location with the largest gradient magnitude is identified based on the gradient vector and is thus the most vulnerable to attack. The selection method is as follows: ， in, i * The most vulnerable position, ∇ L ( S ) i The first element in the gradient vector of the loss function i The gradient magnitude of a malicious act. n This represents the number of malicious behaviors in the original malicious sequence.

5. The adversarial example generation method according to claim 1, characterized in that, The Q-value function is represented using the Bellman equation: ， in, r For the reward function, γ As a discount factor, s t+1 Take action for the sequence to be modified a t The state after that, a t+1 The sequence to be modified is in state s t+1 The chosen action s t The current state of the sequence to be modified. a t For the sequence to be modified in the current state s t The action chosen.

6. The adversarial example generation method according to claim 1, characterized in that, The reward function is constructed based on the loss function of the object detection model, and is expressed as: R ( s,a,s ’ ; φ ) = α • (1 – 2 • L C ( s ’ , y = 0; φ ))， in, s The state of the sequence to be modified. a The sequence to be modified is in state s The chosen action s ’ Take action for the sequence to be modified a The state after that, φ These are the parameters of the target detection model; α This is a scaling factor used to adjust the impact of the loss function on the reward function; L C ( s ’ , y = 0; φ ) is the loss function of the object detection model. y For the true labels of the new sequence, y =0 indicates that the new sequence is a benign sequence.

7. An adversarial sample generation system based on APT group malicious code, characterized in that, The system includes a benign screening module, a perturbation construction module, a location identification module, a sequence generation module, and a code modification module; The benign screening module is used to extract a set of benign subsequences from benign code using a sequence pattern mining algorithm; The perturbation construction module is used to construct a directed graph for each benign subsequence based on the set of benign subsequences, thereby obtaining a set of perturbation actions; The location identification module is used to input the original malicious sequence into the target detection model for detection, and to identify the most vulnerable position in the original malicious sequence by calculating the gradient of the loss function of the target detection model. The sequence generation module is used to select benign subsequences from the perturbation action set and insert them at the most vulnerable positions. It then uses a deep Q-network algorithm to find the best-performing inserted subsequence to obtain the adversarial sequence. Specifically: The original malicious sequence is used as the sequence to be modified. The state of the sequence to be modified is defined as the combination of the sequence to be modified and the most vulnerable position. The action is defined as selecting a benign behavior from the benign subsequence to be inserted and inserting it into the most vulnerable position of the sequence to be modified. The state of the sequence to be modified is input into a deep Q-network algorithm, and a new sequence is obtained by selecting actions using a greedy strategy through the Q-value update function. Simultaneously, the new sequence is input into the object detection model for prediction, and the reward function of the Q-value function is updated based on the loss function of the object detection model; A heuristic backtracking search algorithm is used to constrain the new sequence, specifically: Obtain the directed graph of the benign subsequence to be inserted, calculate the weight of the benign behavior in the benign subsequence to be inserted at the most vulnerable position and sort them; If the new sequence is predicted to be benign by the target detection model, but the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of the benign subsequence to be inserted, then backtrack to the previous state of the new sequence and select the benign behavior with the highest weight from the successor nodes of the benign behavior in the previous state of the new sequence for insertion. When a new sequence is classified as malicious by the target detection model, and the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of the benign subsequence to be inserted, another benign subsequence is selected from the perturbation action set as the benign subsequence to be inserted, and the action selection is restarted from the original malicious sequence. Repeatedly perform action selection, prediction, and constraint until all benign behaviors of the benign subsequence to be inserted have been inserted or the reward function reaches the set threshold, to obtain the optimal new sequence and thus obtain the adversarial sequence; The code modification module is used to apply adversarial sequences to actual malicious code, compile the malicious code into an intermediate representation using the LLVM toolchain, and insert system calls or API calls at predetermined locations in the malware source code to obtain adversarial samples.

8. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the adversarial example generation method according to any one of claims 1-6.

Images