Adversarial sample generation method for organizing malicious codes based on APT

Abstract

The invention discloses an adversarial sample generation method for organizing malicious codes based on APT. The adversarial sample generation method comprises the following steps: firstly, extracting a benign sub-sequence set from benign codes by using a sequence pattern mining algorithm; constructing a directed graph of each benign sub-sequence to obtain a disturbance action set; then inputting the original malicious sequence into a target detection model for detection, and identifying a position which is most easily attacked in the original malicious sequence; then selecting a benign sub-sequence from the disturbance action set, inserting the benign sub-sequence at a position most susceptible to attack, and finding the inserted optimal sub-sequence by using a deep Q network algorithm to obtain a confrontation sequence; and finally, applying the adversarial sequence to an actual malicious code to obtain an adversarial sample. According to the method, the disturbance action set is constructed and the deep Q network algorithm is optimized and inserted, so that the generated adversarial sample can successfully bypass an existing detection model, the original function of a malicious code is reserved, and the attack concealment is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of malware detection, and in particular relates to a method for generating adversarial samples based on APT organization malicious codes. Background Art

[0002] In recent years, the field of network security has faced increasingly severe challenges. Among them, APT (Advanced Persistent Threat) organizations have posed serious threats to enterprises, government agencies and other important organizations through highly complex and customized malicious codes. APT attacks are usually characterized by long-term latency, strong concealment and targeted attacks. Traditional security defense measures are often difficult to deal with these advanced threats.

[0003] Existing malicious code detection methods mainly rely on static feature analysis and signature-based detection technology. Static analysis methods identify malware by checking the signature and structural information of files. However, attackers can easily evade detection by static analysis through code obfuscation, packing, encryption, etc., which makes these methods perform poorly when facing modern advanced malware. Although dynamic analysis can capture the behavioral characteristics of malicious code at runtime, it also faces the challenge of evasion technology. For example, malicious code can identify the virtual environment and change its behavior through environment detection and dynamic loading technology to evade detection.

[0004] Existing detection systems often rely on known malware feature libraries, which contain signature codes or behavior patterns of known malware. However, APT organizations often dynamically adjust their attack methods according to the target environment, which makes it difficult for traditional detection methods based on feature libraries to deal with new and unknown threats. In recent years, with the widespread application of machine learning and deep learning technologies, researchers have tried to use these technologies to improve the efficiency and accuracy of malware detection. However, existing machine learning-based detection methods usually only focus on a certain type of malware feature (such as static features or dynamic behavior), or tend to build a single classification model. These methods may perform well in specific situations, but their performance is often unsatisfactory when dealing with actual environments such as uneven distribution of malware family samples, insufficient sample size, and changing attack methods.

[0005] In summary, existing malware detection technologies have significant deficiencies in dealing with advanced threats from APT organizations. Traditional technologies are difficult to quickly adapt to changes in malicious code and complex attack methods, and lack effective detection capabilities for new threats. Therefore, it is urgent to develop a new method that can dynamically generate adversarial samples, simulate and predict potential variants and attack behaviors of malicious code, thereby improving the robustness and adaptability of the detection system. This is exactly the problem that the present invention is committed to solving. Summary of the invention

[0006] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, provide an adversarial sample generation method based on the malicious code of APT organizations, use the deep Q network algorithm to generate adversarial samples in real time, simulate and predict the attack behavior of APT organizations, thereby enhancing the robustness and response capabilities of network defense systems and models.

[0007] In order to achieve the above object, the present invention provides, on one hand, a method for generating adversarial samples based on malicious code of an APT organization, comprising the following steps:

[0008] Use sequential pattern mining algorithm to extract benign subsequence sets from benign codes;

[0009] According to the benign subsequence set, a directed graph of each benign subsequence is constructed to obtain a perturbation action set;

[0010] The original malicious sequence is input into the target detection model for detection, and the most vulnerable position in the original malicious sequence is identified by calculating the gradient of the loss function of the target detection model;

[0011] Select a benign subsequence from the perturbation action set and insert it at the most vulnerable position. Use the deep Q network algorithm to find the best subsequence to insert and obtain the adversarial sequence.

[0012] The adversarial sequence is applied to the actual malicious code, the malicious code is compiled into an intermediate representation through the LLVM tool chain, and system calls or API calls are inserted at predetermined locations in the malware source code to obtain adversarial samples.

[0013] As a preferred technical solution, the sequence pattern mining algorithm is used to extract a benign subsequence set from the benign code, specifically:

[0014] Obtain a set of benign behavior sequences in existing benign codes;

[0015] Set the minimum support threshold and the empty benign subsequence set;

[0016] The frequency of each benign behavior sequence in the benign behavior sequence set is obtained from the benign behavior sequence set. If the frequency is greater than the minimum support threshold, the sequence is added to the benign subsequence set.

[0017] As a preferred technical solution, for a benign subsequence B containing m different benign behaviors i = {b i1 ,b i2 ,...,b ik ,...,b im},

[0018] The benign subsequence B iConstruct a directed graph G with the benign behaviors in it as nodes and the sequential relationships between the benign behaviors as edges. i , then for the directed graph G i , the node set N i = B i , and the edge set E i = {(b ik , b ik+1 ) | 1 ≤ k < m}, where b ik is the k-th benign behavior in the i-th benign subsequence, b ik+1 is the (k + 1)-th benign behavior in the i-th benign subsequence, and (b ik , b ik+1 ) is the edge between the benign behaviors b ik and b ik+1 ;

[0019] Obtain the adjacency matrix C i of the directed graph G i :

[0020]

[0021] where, e ij represents the sequential relationship between the benign behaviors b ix and b iy , that is, when the benign behavior b iy appears before the benign behavior b ix and there is an edge between them, then e ij = 1, otherwise e ij = 0; b ix is the x-th benign behavior in the i-th benign subsequence, and b iy is the y-th benign behavior in the i-th benign subsequence.

[0022] As a preferred technical solution, identifying the most vulnerable position in the original malicious sequence specifically includes:

[0023] For a given original malicious sequence S = {S1, S2,..., S i ,..., S n}, input it into the target detection model to obtain the gradient vector of the loss function:

[0024]

[0025] where, is the gradient vector of the loss function, S i is the i-th malicious behavior in the original malicious sequence, is the gradient magnitude of the i-th malicious behavior in the original malicious sequence, and n is the number of malicious behaviors in the original malicious sequence;

[0026] According to the gradient vector, find the position with the maximum gradient magnitude as the most vulnerable position. The selection method is:

[0027]

[0028] Among them, i * The most vulnerable position, is the gradient magnitude of the i-th malicious behavior in the gradient vector of the loss function, and n is the number of malicious behaviors in the original malicious sequence.

[0029] As a preferred technical solution, the antagonistic sequence is obtained as follows:

[0030] The original malicious sequence is used as the sequence to be modified, and the state of the sequence to be modified is defined as a combination of the benign subsequence to be inserted, the sequence to be modified, and the most vulnerable position; the action is defined as selecting a benign behavior from the benign subsequence to be inserted and inserting it into the most vulnerable position of the sequence to be modified;

[0031] The state of the sequence to be modified is input into the deep Q network algorithm, and the new sequence is obtained by updating the Q value function and selecting actions using the ∈ greedy strategy;

[0032] At the same time, the new sequence is input into the target detection model for prediction, and the reward function of the Q-value function is updated based on the loss function of the target detection model;

[0033] A heuristic backtracking search algorithm is used to constrain the new sequence;

[0034] Action selection, prediction, and constraints are performed repeatedly until all benign behaviors to be inserted into the benign subsequence have been inserted or the reward function reaches the set threshold, and the optimal new sequence is obtained, thereby obtaining the adversarial sequence.

[0035] As a preferred technical solution, the Q value function is expressed using the Bellman equation:

[0036]

[0037] Among them, r is the reward function, γ is the discount factor, and s t+1 The state of the sequence to be modified after taking action a. t+1 The sequence to be modified is in state s t+1 The action selected, s t is the current state of the sequence to be modified, a t The sequence to be modified is in the current state s t Select the action...

[0038] As a preferred technical solution, the reward function is constructed based on the loss function of the target detection model, which is expressed as:

[0039]

[0040] Among them, s is the state of the sequence to be modified, a is the action selected by the sequence to be modified in state s, and s ’ The state of the sequence to be modified after taking action a. is the parameter of the target detection model; α is the scaling factor, which is used to adjust the impact of the loss function on the reward function; is the loss function of the target detection model, y is the true label of the new sequence, and y = 0 means that the new sequence is a benign sequence.

[0041] As a preferred technical solution, the heuristic backtracking search algorithm is used to constrain the new sequence, specifically:

[0042] Obtain a directed graph of the benign subsequence to be inserted, calculate the weights of the benign behaviors at the most vulnerable positions in the benign subsequence to be inserted, and sort them;

[0043] If the new sequence is predicted to be benign by the target detection model, but the benign behavior inserted in the new sequence does not satisfy the order relationship in the directed graph of the benign subsequence to be inserted, then backtrack to the previous state of the new sequence and select the benign behavior with the highest weight from the successor node of the benign behavior in the previous state of the new sequence for insertion;

[0044] When a new sequence is classified as malicious by the target detection model, and the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of benign subsequences to be inserted, another benign subsequence is selected from the perturbation action set as the benign subsequence to be inserted, and the original malicious sequence is traced back to restart the action selection.

[0045] On the other hand, the present invention provides an adversarial sample generation system based on malicious code of an APT organization, characterized in that the system includes a benign screening module, a perturbation construction module, a location identification module, a sequence generation module and a code modification module;

[0046] The benign screening module is used to extract a benign subsequence set from benign codes using a sequence pattern mining algorithm;

[0047] The perturbation construction module is used to construct a directed graph of each benign subsequence according to the benign subsequence set to obtain a perturbation action set;

[0048] The position identification module is used to input the original malicious sequence into the target detection model for detection, and identify the most vulnerable position in the original malicious sequence by calculating the loss function gradient of the target detection model;

[0049] The sequence generation module is used to select a benign subsequence from the perturbation action set and insert it at the most vulnerable position, and use the deep Q network algorithm to find the most excellent subsequence to be inserted to obtain the adversarial sequence;

[0050] The code modification module is used to apply the adversarial sequence to the actual malicious code, compile the malicious code into an intermediate representation through the LLVM tool chain, and insert a system call or API call at a predetermined location in the malware source code to obtain an adversarial sample.

[0051] Another aspect of the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the above-mentioned adversarial sample generation method.

[0052] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0053] 1. Dynamically generate adversarial samples:

[0054] The present invention utilizes the deep Q-network algorithm to dynamically generate adversarial samples, making the target detection model more adaptable and robust, and being able to effectively deal with advanced threats from APT organizations.

[0055] 2. Effectively bypass detection mechanisms:

[0056] By constructing a perturbation action set and combining it with an optimized insertion strategy, the adversarial samples generated by the present invention can successfully bypass existing detection models while retaining the original functions of the malicious code, thereby enhancing the concealment of the attack.

[0057] 3. Intelligence and efficiency:

[0058] The present invention realizes the intelligent generation of adversarial samples through an automated code modification process, improves the efficiency of the generation process, and reduces the time and resources required for manual intervention.

[0059] 4. Logical rationality and semantic validity:

[0060] A backtracking search algorithm is used to ensure that the generated adversarial samples are logically reasonable and semantically valid, avoiding the problems of function loss and unreasonable code insertion that occur in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 This is an overall flow chart of the adversarial sample generation method based on APT organization malicious code in an embodiment of the present invention.

[0063] Figure 2It is a structural diagram of the adversarial sample generation system based on APT organization malicious code in an embodiment of the present invention.

[0064] Figure 3 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0066] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0067] like Figure 1 As shown, this embodiment provides a method for generating adversarial samples based on malicious code of an APT organization, comprising the steps of:

[0068] S1. Use sequential pattern mining algorithm to extract a set of benign subsequences from benign codes.

[0069] In the process of adversarial sample generation, data mining is a key step. It is necessary to mine representative features from existing benign codes in order to construct subsequent perturbation sets. In this step, a sequence pattern mining algorithm is used to extract representative patterns from benign codes to obtain a set of benign subsequences, specifically:

[0070] S101, obtaining a set of benign behavior sequences in existing benign codes;

[0071] S102, setting a minimum support threshold and an empty benign subsequence set;

[0072] S103: Obtain the frequency of occurrence of each benign behavior sequence in the benign behavior sequence set from the benign behavior sequence set. If the frequency is greater than a minimum support threshold, add it to the benign subsequence set.

[0073] The benign subsequences obtained through this mining method will better reflect the behavior patterns of benign codes, making them more representative.

[0074] S2. Construct a directed graph for each benign subsequence based on the set of benign subsequences to obtain a set of perturbation actions.

[0075] Construct a set of perturbation actions from the extracted set of benign subsequences, and use a directed graph G to represent the sequential relationship of each benign subsequence. The set of perturbation actions implies the range of possible perturbations that can be introduced by selectively inserting various benign behavior segments into the original malicious behavior sequence, thereby enabling the perturbed sequence to confuse the target detection model.

[0076] Further, for a benign subsequence B i ={b i1 , b i2 ,..., b ik ,..., b im} containing m different benign behaviors, use the benign behaviors in the benign subsequence B i as nodes and the sequential relationship between the benign behaviors as edges to construct a directed graph G i . Then, the node set N i of the directed graph G i = B i , and the edge set E i = {(b ik , b ik+1 ) | 1 ≤ k < m}, where b ik is the k-th benign behavior in the i-th benign subsequence, b ik+1 is the (k + 1)-th benign behavior in the i-th benign subsequence, and (b ik , b ik+1 ) is the edge between the benign behaviors b ik and b ik+1 ; then obtain the adjacency matrix C i of the directed graph G i of the i-th benign subsequence:

[0077]

[0078] where, e ij represents the sequential relationship between the benign behaviors b ix and b iy , that is, when the benign behavior b iy appears after the benign behavior b ix and there is an edge between them, then e ij = 1, otherwise e ij = 0; b ix is the x-th benign behavior in the i-th benign subsequence, and b iy is the y-th benign behavior in the i-th benign subsequence. The adjacency matrix is used to describe the allowed order between benign behaviors, rather than the relationship between benign behaviors.

[0079] By starting the directed graph traversal from different initial nodes and selecting successor nodes according to the structure of the graph, different behavior sequences can be generated, which can introduce a wider range of sequence changes while satisfying the sequence relationship between benign behaviors. Inserting these benign sequence fragments into malicious sequences helps to improve the complexity and semantic effectiveness of adversarial samples. For example, consider the following directed graph, which contains five benign behaviors: b1, b2, b3, b4, and b5, and has six benign behavior sequence relationships: b1→b2, b1→b3, b2→b4, b2→b5, b3→b4, b4→b5. By traversing this directed graph, the following possible benign behavior sequences can be generated: starting from b1, traversing the graph obtains the sequence (b1, b2), (b1, b2, b4), (b1, b2, b4, b5), (b1, b3), (b1, b3, b4), (b1, b3, b4, b5). Similarly, starting from b2, traversing the graph obtains the sequences (b2, b4) and (b2, b4, b5), which demonstrates the flexibility and ability of directed graphs to represent behavior patterns and generate various benign behavior sequences.

[0080] S3. Input the original malicious sequence into the target detection model for detection, and identify the most vulnerable position in the original malicious sequence by calculating the gradient of the loss function of the target detection model.

[0081] Furthermore, the gradient of the loss function of the target detection model relative to the position in the sequence can be used to find the most effective modification position, rather than randomly modifying the behavior sequence; the position with a higher gradient value has a more significant impact on the loss function, indicating that it is easy to modify. This method ensures that the modifications made to the malicious behavior sequence have the greatest impact on its classification results. Specifically:

[0082] For a given original malicious sequence S = {S1, S2, ..., S i ,...,S n}, input the target detection model to get the gradient vector of the loss function:

[0083]

[0084] in, is the gradient vector of the loss function, S i is the i-th malicious behavior in the original malicious sequence, is the gradient amplitude of the i-th malicious behavior in the original malicious sequence, and n is the number of malicious behaviors in the original malicious sequence;

[0085] According to the gradient vector, find the position with the maximum gradient magnitude as the most vulnerable position. The selection method is:

[0086]

[0087] Among them, i * The most vulnerable position, is the gradient magnitude of the i-th malicious behavior in the gradient vector of the loss function, and n is the number of malicious behaviors in the original malicious sequence.

[0088] S4. Select a benign subsequence from the perturbation action set and insert it at the most vulnerable position. Use the deep Q network algorithm to find the most optimal subsequence to obtain the adversarial sequence.

[0089] Furthermore, this step determines the modification strategy at the position in the sequence that is most susceptible to disturbance by learning the abnormal behavior characteristics of the sequence, and finally generates adversarial samples. This application uses the deep Q network (DQN) algorithm to guide the modification process, with the goal of iteratively modifying the original malicious sequence until the target detection model classifies it as benign. Specifically:

[0090] S401. The original malicious sequence is used as the sequence to be modified, and the state of the sequence to be modified is defined as a combination of a benign subsequence to be inserted, the sequence to be modified, and the most vulnerable position; an action is defined as randomly selecting a benign subsequence from a perturbation action set and inserting it into the most vulnerable position of the sequence to be modified.

[0091] More specifically, the state is represented by a sequence of behaviors, i.e., the sequence of behaviors after the benign subsequence to be inserted is inserted at the most vulnerable position of the sequence to be modified; each sequence is encoded using a one-hot encoding method, where each value corresponds to a specific behavior. The state vector s∈S* is a sequence of behaviors of length n, where each element of the vector s corresponds to a specific behavior, represented as a number s i , and s i ∈R, where 1≤i≤n; the complete state S* can be expressed as: S*={s|s=(s1,s2,...,s n ),s i ∈R,1≤i≤n}.

[0092] The state is processed by the target detection model and the DQN algorithm. The target detection model simulates black box detection by submitting specific inputs to obtain detection results. Since most current target detection models are black box models, the adversarial sequence generated by this application is based on the target detection model to perform adversarial attacks. The DQN model is designed to modify malicious sequences to mislead the target detection model.

[0093] Actions represent possible modification operations that can be applied to the original malicious sequence. In order to maintain the original functionality of the program, the perturbation process is limited to only inserting benign behaviors into the malicious sequence while maintaining its original malicious functionality. n} represents the benign subsequence B corresponding to the operation = {b1, b2, ..., b m} Insert the action at the most vulnerable position in the original malicious sequence. For each action a i ∈A, which can be expressed as: Insert(S,b i ,p)→S ’ =[s1,...,s p-1 ,b i ,s p ,...,s k ];

[0094] Among them, S is the original malicious sequence with k malicious behaviors, S ’ is the new sequence, b i is the inserted benign behavior, p is the position where the benign behavior is inserted, and b i After inserting at position p, the new sequence S ’ Now contains k+1 behaviors.

[0095] S402: Input the state of the sequence to be modified into the deep Q network algorithm, and obtain a new sequence by updating the Q value function and selecting actions using the ∈ greedy strategy.

[0096] The DQN algorithm uses reinforcement learning to optimize the modification effect. The modification process of each step can be expressed as s t+1 =M(s t ,a t ), where M is a t The modification action is applied to the sequence state s to be modified t The function then uses the Bellman equation to evaluate the Q-value of taking a specific action in a specific state:

[0097]

[0098] Among them, r is the reward function, γ is the discount factor, and s t+1 The state of the sequence to be modified after taking action a. t+1 The sequence to be modified is in state s t+1 The action selected, s t is the current state of the sequence to be modified, a t The sequence to be modified is in the current state s t Selected Actions. To select the most effective attack action at each step, the DQN algorithm uses Q-values, which represent the expected long-term reward of a specific action in the current state.

[0099] S403. At the same time, the new sequence is input into the target detection model for prediction, and the reward function of the Q-value function is updated based on the loss function of the target detection model.

[0100] During the training process, DQN optimizes the modification strategy by observing the detection results of the target detection model on the modified sequence and obtains corresponding rewards. The present invention uses the reward function to guide the DQN algorithm to effectively modify the original malicious sequence that affects its classification results. The reward function is constructed based on the loss function of the target detection model and is expressed as:

[0101]

[0102] Among them, s is the state of the sequence to be modified, a is the action selected by the sequence to be modified in state s, and s ’ The state of the sequence to be modified after taking action a. is the parameter of the target detection model; α is the scaling factor, which is used to adjust the impact of the loss function on the reward function; is the loss function of the target detection model, y is the true label of the new sequence, and y = 0 means that the new sequence is a benign sequence (i.e., normal class). By using this reward function, the DQN algorithm can be guided to minimize the loss of the normal class (label 0) proxy model, thereby effectively modifying the original malicious sequence to make it closer to the benign sequence.

[0103] In this embodiment, the prediction function of the target detection model is assumed to be , where s represents the current state, Corresponding to the parameters of the target detection model; since the target detection model is a binary classifier, the loss function L C It can be expressed as categorical cross entropy loss:

[0104]

[0105] Among them, y is the true label of the new sequence, It represents the predicted probability that the new sequence is predicted to be the i-th category.

[0106] S404, using a heuristic backtracking search algorithm to constrain the new sequence.

[0107] More specifically, by training the DQN algorithm to learn the optimal strategy for insertion behavior, the ability of the DQN algorithm to predict the optimal insertion behavior is enhanced, and the action value function Q(s, a; θ) continuously approaches the optimal action value function Q * (s,a). When using the DQN algorithm to predict and prioritize available actions, the action that has the greatest impact on the classifier decision is selected at each step:

[0108]

[0109] However, the DQN algorithm may not be able to account for the dependencies between inserted behaviors when selecting the sequence of inserted behaviors. For example, suppose there is a set of behaviors B = {b1, b2, b3}, and their corresponding dependencies are defined by rule R. According to R, b1 must be followed by b2, and b2 must be followed by b3 to maintain the dependency constraints. However, the algorithm calculates the action estimate function Q(s t ,a;θ t ), which is equivalent to solving the following optimization problem for each behavior:

[0110]

[0111] This independent behavior choice may cause the sequence to violate the dependency constraints specified by rule R. The DQN algorithm may choose to insert b1 and b3 into the original sequence without inserting b2, resulting in an infeasible sequence: infeasiblesequence = {..., b1,..., b3,...}. To solve this problem, this application adopts a heuristic backtracking search algorithm, which systematically tries and discards non-optimal solutions to obtain the optimal solution, specifically:

[0112] Obtain a directed graph of the benign subsequence to be inserted, calculate the weights of the benign behaviors at the most vulnerable positions in the benign subsequence to be inserted, and sort them;

[0113] If the new sequence is predicted to be benign by the target detection model, but the benign behavior inserted in the new sequence does not satisfy the order relationship in the directed graph of the benign subsequence to be inserted, then backtrack to the previous state of the new sequence and select the benign behavior with the highest weight from the successor node of the benign behavior in the previous state of the new sequence for insertion;

[0114] When a new sequence is classified as malicious by the target detection model, and the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of benign subsequences to be inserted, another benign subsequence is selected from the perturbation action set as the benign subsequence to be inserted, and the action selection heuristic backtracking search algorithm is re-started by backtracking to the original malicious sequence, using a continuous process to try to modify the malicious sequence while complying with all behavioral dependency constraints. Based on the perturbation action set represented by the directed graph, when traversing various modification paths, if the generated adversarial sample cannot successfully evade the classifier, it returns to the early state and explores alternative options; through this iterative trial-and-error method, adversarial samples that meet all behavioral dependency constraints can be generated.

[0115] During the iteration of the main loop of the DQN algorithm, the weights of the positions inserted into the sequence are reordered, and the subsequent nodes are obtained by traversing all nodes in the directed graph. If the inserted behavior meets the dependency relationship specified by the directed graph, the malware behavior sequence modified by the behavior is allowed to be predicted by the model. If the new sequence is classified as benign, the algorithm ends successfully; if the new sequence is classified as malicious, the search changes the next directed graph, allowing other behaviors to be inserted into the sequence. This process embodies the backtracking characteristics of the algorithm. If a malicious behavior sequence cannot be modified to a benign classification, try to recover its decision and explore alternative sequences. The heuristic backtracking search algorithm prioritizes promising paths and the adaptability of backtracking, reverses invalid decisions, and aims to discover a series of behaviors that meet dependency constraints and obtain the desired benign classification.

[0116] S405, repeatedly performing action selection, prediction and constraint until all benign behaviors to be inserted into the benign subsequence have been inserted or the reward function reaches a set threshold, obtaining an optimal new sequence and then obtaining an adversarial sequence.

[0117] Step S4 selects the optimal insertion behavior based on the original malicious sequence and the perturbation action set through the DQN algorithm. The algorithm uses a deep neural network as a DQN agent to estimate the Q function, which outputs the Q values ​​of different actions and guides the agent to select the action that maximizes the expected reward. In each cycle, the agent model selects an action to execute according to the greedy (ε-greedy) strategy, observing the reward and the next state. The agent then learns to maximize the expected cumulative reward, guiding its action selection towards the sequence modification that produces the highest reward. At the same time, DQN has two key mechanisms: experience replay and target network. Experience replay allows learning from past experience and enhances stability by breaking correlations in the data. The target network provides a stable target for Q function updates, and each sequence is iteratively modified until it is considered benign or the maximum number of modification attempts is reached.

[0118] S5. Apply the adversarial sequence to the actual malicious code, compile the malicious code into an intermediate representation through the LLVM tool chain, and insert system calls or API calls at predetermined locations in the malware source code to obtain adversarial samples.

[0119] The main task of this step is to apply the perturbation operation determined in the previous step to the actual malicious code to generate adversarial samples that can evade detection in a real environment. First, compile the malicious code into an LLVM intermediate representation (IR) file with debugging information. This format facilitates subsequent code analysis and modification. After generating the LLVM IR file, use the strace tool to obtain the system call sequence of the malicious code at runtime, and map these system calls to specific source code locations through the addr2line tool. Next, insert predetermined system calls or API calls at these determined code locations to achieve effective perturbations to the behavior sequence. Through such automated modifications, not only can the generated adversarial samples be guaranteed to be logically reasonable, but they can also be guaranteed to have the original malicious functions and be successfully executed in the target system.

[0120] Finally, the modified IR file is recompiled to generate an executable file. This process ensures that the perturbation operation of the malicious code is correctly applied and generates a system call sequence consistent with expectations during actual operation. This makes it possible to verify whether these modifications are effective, that is, whether the generated adversarial samples can evade existing detection models while maintaining their original malicious functions. This step makes it possible to apply adversarial samples from theoretical design to practical application, enhancing the defense capabilities against malicious code of APT organizations.

[0121] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0122] Based on the same idea as the adversarial sample generation method based on APT organization malicious code in the above-mentioned embodiment, the present invention also provides an adversarial sample generation system based on APT organization malicious code, which can be used to execute the above-mentioned adversarial sample generation method based on APT organization malicious code. For ease of explanation, the structural diagram of the adversarial sample generation system embodiment based on APT organization malicious code only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0123] like Figure 2 As shown, another embodiment of the present invention provides an adversarial sample generation system based on APT organization malicious code, including a benign screening module, a perturbation construction module, a location identification module, a sequence generation module and a code modification module;

[0124] Among them, the benign screening module is used to extract a benign subsequence set from benign codes using a sequence pattern mining algorithm;

[0125] The perturbation construction module is used to construct a directed graph of each benign subsequence according to the benign subsequence set to obtain a perturbation action set;

[0126] The position identification module is used to input the original malicious sequence into the target detection model for detection, and identify the most vulnerable position in the original malicious sequence by calculating the gradient of the loss function of the target detection model;

[0127] The sequence generation module is used to select benign subsequences from the perturbation action set and insert them at the most vulnerable position. The deep Q network algorithm is used to find the most excellent subsequence to be inserted to obtain the adversarial sequence.

[0128] The code modification module is used to apply the adversarial sequence to the actual malicious code, compile the malicious code into an intermediate representation through the LLVM tool chain, and insert system calls or API calls at predetermined locations in the malware source code to obtain adversarial samples.

[0129] It should be noted that the adversarial sample generation system based on APT organization malicious code of the present invention corresponds one-to-one to the adversarial sample generation method based on APT organization malicious code of the present invention. The technical features and beneficial effects described in the embodiment of the adversarial sample generation method based on APT organization malicious code are applicable to the embodiment of the adversarial sample generation system based on APT organization malicious code. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. This is hereby declared.

[0130] In addition, in the implementation of the adversarial sample generation system based on the APT organization malicious code in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the adversarial sample generation system based on the APT organization malicious code is divided into different program modules to complete all or part of the functions described above.

[0131] like Figure 3 As shown, in one embodiment, a computer-readable storage medium is provided, which stores a program in a memory. When the program is executed by a processor, a method for generating adversarial samples based on malicious code of an APT organization is implemented, specifically:

[0132] Use sequential pattern mining algorithm to extract benign subsequence sets from benign codes;

[0133] According to the benign subsequence set, a directed graph of each benign subsequence is constructed to obtain a perturbation action set;

[0134] The original malicious sequence is input into the target detection model for detection, and the most vulnerable position in the original malicious sequence is identified by calculating the gradient of the loss function of the target detection model;

[0135] Select a benign subsequence from the perturbation action set and insert it at the most vulnerable position. Use the deep Q network algorithm to find the best subsequence to insert and obtain the adversarial sequence.

[0136] The adversarial sequence is applied to the actual malicious code, the malicious code is compiled into an intermediate representation through the LLVM tool chain, and system calls or API calls are inserted at predetermined locations in the malware source code to obtain adversarial samples.

[0137] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0138] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. The adversarial sample generation method based on the malicious code of the APT organization is characterized by: The steps include: Use sequential pattern mining algorithm to extract benign subsequence sets from benign codes; According to the benign subsequence set, a directed graph of each benign subsequence is constructed to obtain a perturbation action set; The original malicious sequence is input into the target detection model for detection, and the most vulnerable position in the original malicious sequence is identified by calculating the gradient of the loss function of the target detection model; Select a benign subsequence from the perturbation action set and insert it at the most vulnerable position. Use the deep Q network algorithm to find the best subsequence to insert and obtain the adversarial sequence. The adversarial sequence is applied to the actual malicious code, the malicious code is compiled into an intermediate representation through the LLVM tool chain, and system calls or API calls are inserted at predetermined locations in the malware source code to obtain adversarial samples.

2. The adversarial sample generation method according to claim 1, characterized in that: The method of extracting a benign subsequence set from benign codes using a sequence pattern mining algorithm is as follows: Obtain a set of benign behavior sequences in existing benign codes; Set the minimum support threshold and the empty benign subsequence set; The frequency of each benign behavior sequence in the benign behavior sequence set is obtained from the benign behavior sequence set. If the frequency is greater than the minimum support threshold, the sequence is added to the benign subsequence set.

3. The adversarial sample generation method according to claim 1, characterized in that: For a benign subsequence B containing m different benign behaviors i = {b i1 ,b i2 ,...,b ik ,...,b im }, Take the benign behaviors in the benign subsequence B i as nodes, and the sequential relationship between the benign behaviors as edges to construct a directed graph G i . Then, for the directed graph G i , the node set N i = B i , and the edge set E i = {(b ik , b ik+1 ) | 1 ≤ k < m}, where b ik is the k-th benign behavior in the i-th benign subsequence, b ik+1 is the (k + 1)-th benign behavior in the i-th benign subsequence, and (b ik , b ik+1 ) is the edge between the benign behaviors b ik and b ik+1 ; Get the directed graph G i The adjacency matrix C i : Among them, e ij Indicates good behavior ix With b iy The order relationship between them, that is, when the benign behavior b iy Appears in benign behavior ix Then, if there is an edge between them, then e ij =1, otherwise e ij =0; b ix is the xth benign behavior in the ith benign subsequence, b iy is the yth benign behavior in the ith benign subsequence.

4. The adversarial sample generation method according to claim 1, characterized in that: The identification of the most vulnerable position in the original malicious sequence is specifically: For a given original malicious sequence S = {S1, S2, ..., S i ,...,S n }, input the target detection model to get the gradient vector of the loss function: in, is the gradient vector of the loss function, S i is the i-th malicious behavior in the original malicious sequence, is the gradient amplitude of the i-th malicious behavior in the original malicious sequence, and n is the number of malicious behaviors in the original malicious sequence; According to the gradient vector, find the position with the maximum gradient magnitude as the most vulnerable position. The selection method is: Among them, i * The most vulnerable position, is the gradient magnitude of the i-th malicious behavior in the gradient vector of the loss function, and n is the number of malicious behaviors in the original malicious sequence.

5. The adversarial sample generation method according to claim 1, characterized in that: The obtained confrontation sequence is specifically: The original malicious sequence is used as the sequence to be modified, and the state of the sequence to be modified is defined as a combination of the benign subsequence to be inserted, the sequence to be modified, and the most vulnerable position; the action is defined as selecting a benign behavior from the benign subsequence to be inserted and inserting it into the most vulnerable position of the sequence to be modified; The state of the sequence to be modified is input into the deep Q network algorithm, and the new sequence is obtained by updating the Q value function and selecting actions using the ∈ greedy strategy; At the same time, the new sequence is input into the target detection model for prediction, and the reward function of the Q-value function is updated based on the loss function of the target detection model; A heuristic backtracking search algorithm is used to constrain the new sequence; Action selection, prediction, and constraints are performed repeatedly until all benign behaviors to be inserted into the benign subsequence have been inserted or the reward function reaches the set threshold, and the optimal new sequence is obtained, thereby obtaining the adversarial sequence.

6. The adversarial sample generation method according to claim 5, characterized in that: The Q value function is expressed using the Bellman equation: Among them, r is the reward function, γ is the discount factor, and s t+1 The state of the sequence to be modified after taking action a. t+1 The sequence to be modified is in state s t+1 The action selected, s t is the current state of the sequence to be modified, a t The sequence to be modified is in the current state s t Select the action.

7. The adversarial sample generation method according to claim 5, characterized in that: The reward function is constructed based on the loss function of the target detection model and is expressed as: Among them, s is the state of the sequence to be modified, a is the action selected by the sequence to be modified in state s, and s ’ The state of the sequence to be modified after taking action a. is the parameter of the target detection model; α is the scaling factor, which is used to adjust the impact of the loss function on the reward function; is the loss function of the target detection model, y is the true label of the new sequence, and y = 0 means that the new sequence is a benign sequence.

8. The adversarial sample generation method according to claim 5, characterized in that: The heuristic backtracking search algorithm is used to constrain the new sequence, specifically: Obtain a directed graph of the benign subsequence to be inserted, calculate the weights of the benign behaviors at the most vulnerable positions in the benign subsequence to be inserted, and sort them; If the new sequence is predicted to be benign by the target detection model, but the benign behavior inserted in the new sequence does not satisfy the order relationship in the directed graph of the benign subsequence to be inserted, then backtrack to the previous state of the new sequence and select the benign behavior with the highest weight from the successor node of the benign behavior in the previous state of the new sequence for insertion; When a new sequence is classified as malicious by the target detection model, and the benign behavior inserted in the new sequence does not satisfy its order relationship in the directed graph of benign subsequences to be inserted, another benign subsequence is selected from the perturbation action set as the benign subsequence to be inserted, and the original malicious sequence is traced back to restart the action selection.

9. The adversarial sample generation system based on the malicious code of APT organization is characterized by: The system includes a benign screening module, a perturbation construction module, a location identification module, a sequence generation module, and a code modification module; The benign screening module is used to extract a benign subsequence set from benign codes using a sequence pattern mining algorithm; The perturbation construction module is used to construct a directed graph of each benign subsequence according to the benign subsequence set to obtain a perturbation action set; The position identification module is used to input the original malicious sequence into the target detection model for detection, and identify the most vulnerable position in the original malicious sequence by calculating the loss function gradient of the target detection model; The sequence generation module is used to select a benign subsequence from the perturbation action set and insert it at the most vulnerable position, and use the deep Q network algorithm to find the most excellent subsequence to be inserted to obtain the adversarial sequence; The code modification module is used to apply the adversarial sequence to the actual malicious code, compile the malicious code into an intermediate representation through the LLVM tool chain, and insert a system call or API call at a predetermined location in the malware source code to obtain an adversarial sample.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the adversarial sample generation method described in any one of claims 1 to 8 is implemented.

Images