Reinforcement learning-based anti-attack method for avoiding static malicious software detector
Through the RLAEG framework based on reinforcement learning, the adversarial sample generation environment and action space are optimized, and the problem of insufficient adversarial sample generation efficiency and generalization in the existing technology is solved, and the adversarial sample generation with complete functions is achieved efficiently generated in the black box environment. Bypass different detectors and reveal detector weaknesses.
Patent Information
- Application Number
- CN202411219418.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2025-01-03
AI Technical Summary
The existing attack methods need to know the parameters of the victim model when generating adversarial samples, and it is difficult to effectively generate adversarial samples in a black box environment. The existing methods have shortcomings in sample generation efficiency and generalization and are easily detected.
The RLAEG framework based on reinforcement learning is designed, and the agent is trained through the accompanying functional verification module, feature engineering, Soft-Actor-Critic algorithm and strategy distillation to optimize the adversarial sample generation environment and action space, and improve the generation efficiency and generalization of adversarial sample generation.
It realizes efficient generation of functionally complete adversarial samples in a black box environment, improves the generation efficiency and generalization of adversarial samples, and can effectively bypass different types of malware detectors, revealing the potential weaknesses of the detector.
Smart Images
Figure CN119249418A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of malware, and in particular relates to a reinforcement learning-based counterattack method for evading static malware detectors. Background Art
[0002] With the in-depth study of machine learning technology, traditional signature-based static detection has encountered difficulties in identifying unknown or obfuscated samples. Various detection methods based on machine learning technology have become popular, including byte sequence-based methods, image-based methods, etc. Although this method can detect unknown samples and save time and effort, it also has security risks. Researchers use technologies such as genetic algorithms, reinforcement learning, and adversarial generative networks to generate adversarial samples that evade detection.
[0003] However, current attack methods face a series of challenges. First, many attack methods require knowing the parameters of the victim model to launch an attack, which is the so-called white-box attack. For black-box attacks, especially when only sample labels are available, the effects of current attack methods are far from satisfactory. Second, many attack methods can only generate adversarial examples in the feature space, which can bypass the detector but makes it difficult to generate real and effective adversarial samples in the problem space. Finally, existing sample generation methods rely on a large number of query times and content injection, making the attack easy to detect. Adversarial sample generation methods based on reinforcement learning can partially solve these problems, but they still have problems with manual function verification and inefficiency in sample generation. Summary of the invention
[0004] Purpose of the invention: To solve the above problems, the present invention discloses a RLAEG (Reinforcement Learning-based Adversarial Example Generation) framework for improving the generation efficiency of adversarial samples while ensuring the functional integrity of the samples. In addition, it also highlights the vulnerabilities and limitations of popular malware detectors. In addition, the present invention also adopts a transfer learning-based strategy to extract knowledge from agents trained for different detectors, so as to take specific actions when facing different detectors and improve the versatility of the attack.
[0005] Technical solution: A reinforcement learning-based anti-attack method for evading static malware detectors of the present invention comprises the following steps:
[0006] Step 1: Design an adversarial sample generation environment with a functional verification module;
[0007] Step 2: Design the observation space of the intelligent agent based on feature engineering;
[0008] Step 3: Improve the agent’s action space;
[0009] Step 4: Use the Soft-Actor-Critic algorithm to train the agent to generate adversarial samples;
[0010] Step 5: Use policy distillation to train an agent with stronger attack generalization to fight against different types of detectors.
[0011] Furthermore, in step 1, the adversarial sample generation environment of the design with the function verification module is specifically: the adversarial sample generation environment includes a file analyzer, a feature extractor, a static malware detector based on machine learning, a file modifier, and the original sample to be modified; this task has the following settings:
[0012] The adversary's goal is to generate effective adversarial samples by performing a series of modification operations on the original samples while retaining their functionality to evade various detectors; during the attack, the adversary operates in a black box environment, and they do not know the parameters, gradients, and other information of the victim model, nor the original scores of the samples; the adversary's goal is to generate functional adversarial malware in the problem space; if a sample successfully evades the detector through modification within the specified steps and passes the functional verification, it is regarded as a successful adversarial sample; conversely, if the number of modifications reaches the upper limit, or the sample structure is destroyed or the function is changed during the modification process, the modification is regarded as a failure.
[0013] Furthermore, in the adversarial sample generation process, the file analyzer will analyze the original sample to evaluate the integrity, parsability and maliciousness of the sample structure. If structural damage or parsing problems are detected during the analysis process, the sample will be deemed damaged and excluded from subsequent modification operations. The feature extractor will then extract features from the analyzed sample to derive features for the detector to assess the degree of maliciousness and for the agent to determine appropriate modification operations. The file modification will specify the sample according to the corresponding modification actions output by the agent.
[0014] Furthermore, the file analyzer includes three modules, which analyze the readability, parsability and functional integrity of the sample respectively; first, the file analyzer uses Python's local file reading module to process the sample to ensure the extraction of byte content; then, the file analyzer uses LIEF to parse the sample, extract data for feature construction, and perform subsequent file modifications; LIEF is a platform that can parse files and provides an API for accessing internal data of files; finally, the file analyzer uses Angr to implement verification operations on the retention of functions before and after sample modification; Angr is an open source binary analysis platform that can analyze binary files; it can restore the control flow graph CFG of the binary file; the nodes of CFG represent basic instruction blocks, and the edges represent potential control flow transfers between them; the location and content of the function in the binary file can be determined, and the function of the sample can be further analyzed through the CFG of the sample; therefore, by comparing whether the CFG of the sample has changed before and after modification, it can be verified whether the function of the sample has been retained.
[0015] Furthermore, in step 2, the observation space of the agent is designed based on feature engineering, specifically: the complete features representing the malware are divided into eight features according to their structure, namely, the byte histogram of the sample, the two-dimensional byte entropy histogram, the string information, the general information of the file, the header information of the file, the chapter information of the file, the import table information and the export table information, and these features are optimized by feature engineering; the following parameters are defined:
[0016] x represents the original sample, which is converted into a vector v by the feature extractor f. x , F i Indicates v x The i-th feature among the eight features, v x [F i ][j] represents the j-th dimension sub-feature in the i-th feature, and v' x Represents the modified sample feature vector; further analysis is performed by comparing the average distance change of each group of samples before and after modification. o(α) represents a very small value to prevent illegal division by zero. The average distance change ADV is calculated by the following formula:
[0017]
[0018] v x =f(x)
[0019] The above formula is used to effectively screen the features, and then the GBDT model is used to determine the importance of different features in the original observation data to distinguish adversarial samples; a subset of adversarial examples is obtained by using the original state space to evade the detection of the EMBER detector; these samples are used together with the original samples to train the GBDT detector to distinguish adversarial examples; the observation space of the agent is finally determined through the above two methods.
[0020] Furthermore, in step 3, the action space of the improved intelligent agent is specifically as follows: the action space is improved, and the improved action space is divided into four categories: injection, extension, modification and conversion; the interference is injected into different positions of the original sample through the injection operation; the injected disturbance comes from specific content in the benign file; the extension operation involves moving the specific content in the original sample backward according to the structural characteristics and arrangement of the file, thereby creating space for modification, and introducing two operations in the operation space, namely, extending the file header and the extension part; the modification operation involves changing certain features of the original sample without injecting new disturbances; there are five different modification behaviors for different positions and features of the sample, involving modification operations on the header, chapter table and information at the end of the sample; the conversion operation refers to applying UPX technology to the original sample by packing or unpacking.
[0021] Furthermore, in step 4, the Soft-Actor-Critic algorithm is used to train the intelligent agent to generate adversarial samples. Specifically, the Soft-Actor-Critic algorithm includes the concept of information entropy and is suitable for tasks with large and complex state spaces. At the same time, after the adversarial sample generation is completed, feature engineering and the interpretability of the model are used to explain the generation of the adversarial samples, thereby explaining the weaknesses and deficiencies of the malware detector.
[0022] Furthermore, in step 5, the strategy distillation training is used to attack an agent with stronger generalization to fight against different types of detectors, specifically:
[0023] The proposed transferable attack framework is implemented in two phases: training and testing. In the training phase, the agent is trained by multi-task policy distillation. In the data collection phase, a subset of the original samples is extracted from the training set, and multiple agents trained by the SAC algorithm are used as experts to interact with their respective detectors and generate outputs. These outputs, as well as the input and detector labels, are stored in the replay memory for subsequent updates. In the update phase, the student agent extracts experience from the replay buffer, generates corresponding outputs based on the input and label, calculates the distillation loss, and updates the policy network. Finally, multiple policies from different tasks are merged into one policy.
[0024] In the prediction phase, when the detector is known, the corresponding modification operation is directly obtained by telling the agent the input and the detector label. When the detector is unknown, the agent initially generates all modification operations based on the sample state. Then, the modified samples generated by applying different operations are submitted to the unknown detector for query; if all modified samples fail to evade successfully, the sample with the lowest malicious score is selected as the input of the agent's next round of modification; according to the results of attack experiments against known detectors, the average number of queries used to generate adversarial samples does not exceed four times; therefore, if the sample still cannot evade detection after four modifications, the detector label represented by the sample with the lowest score in each round is used to determine the potential detector that best matches the unknown detector; the specific method is to select the detector with the highest frequency in the four rounds of recorded labels; subsequent modifications only use this label to obtain the corresponding output; consistent with previous experiments, failure to evade detection within the maximum number of modifications or any modification that causes a change in the sample's function is considered a failure.
[0025] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: The present invention utilizes reinforcement learning to observe the characteristics of different malicious samples and perform corresponding modification operations to circumvent malware detectors. The present invention proposes for the first time to incorporate the functional verification operation of the sample into the adversarial sample generation framework as part of the automatic generation of adversarial samples. In addition, the present invention also optimizes the observation space and action space of the reinforcement learning-based agent, greatly improving the generation efficiency of adversarial samples. The present invention also considers the generalization problem of adversarial samples for the first time, and uses the method of multi-task strategy distillation to greatly improve the generalization of adversarial samples. Finally, based on the attack effect of the adversarial samples, the present invention explains the possible defects and shortcomings of currently popular malware detectors. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 Schematic diagram of the proposed reinforcement learning-based adversarial example generation framework with functional verification.
[0027] Figure 2 Schematic diagram of using feature engineering to interpret the observation space of the agent in the proposed framework according to feature importance.
[0028] Figure 3 A schematic diagram of the feature vector used by the intelligent agent after feature engineering.
[0029] Figure 4 Schematic diagram of feature engineering used to explain the gradient boosting decision tree model based on machine learning by feature importance.
[0030] Figure 5Schematic diagram of using the interpretability of deep learning to explain various detector models based on deep neural networks according to feature importance.
[0031] Figure 6 Schematic diagram of the proposed adversarial sample generation framework based on multi-task policy distillation.
[0032] Figure 7 Illustration of the generalization of an agent trained for a specific detector to attacks from other detectors. DETAILED DESCRIPTION
[0033] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0034] The adversarial sample generation framework based on reinforcement learning and with functional verification described in the present invention includes the following steps:
[0035] S1: Design an adversarial sample generation environment with a functional verification module.
[0036] like Figure 1 As shown in the figure, the environment includes a file analyzer, a feature extractor, a static malware detector based on machine learning, a file modifier, and the original sample to be modified. This task has the following settings:
[0037] The adversary's goal is to generate effective adversarial samples by performing a series of modification operations on the original samples while retaining their functionality to evade various detectors. During the attack, the adversary operates in a black box environment, where they do not know the parameters, gradients, and other information of the victim model, nor the original scores of the samples. The adversary's goal is to generate functional adversarial malware within the problem space. If a sample successfully evades the detector through modification within the specified steps and passes the functional verification, it is considered a successful adversarial sample. Conversely, if the number of modifications reaches the upper limit, or the sample structure is destroyed or the function is changed during the modification process, the modification is considered a failure.
[0038] During the adversarial sample generation process, the file analyzer analyzes the original sample to evaluate the integrity, parsability, and maliciousness of the sample structure. If structural damage or parsing problems are detected during the analysis process, the sample will be considered damaged and excluded from subsequent modification operations. The feature extractor will then extract features from the analyzed sample to derive features for the detector to assess the degree of maliciousness and for the agent to determine appropriate modification operations. File modification specifies the sample based on the corresponding modification actions output by the agent.
[0039] The file analyzer in this task environment contains three modules, which analyze the readability, parsability and functional integrity of the sample respectively. First, the file analyzer uses Python's local file reading module to process the sample to ensure the extraction of byte content. After that, the file analyzer uses LIEF to parse the sample, extract data for feature construction, and perform subsequent file modifications. LIEF is a platform that can parse files and provides an API for accessing internal data of files. Finally, the file analyzer uses Angr to implement the verification operation of the function retention before and after the sample is modified. Angr is an open source binary analysis platform that can analyze binary files. It can restore the control flow graph (CFG) of the binary file. CFG plays an important role in malware analysis, where nodes represent basic instruction blocks and edges represent potential control flow transfers between them. The location and content of the function in the binary file can be determined, and the function of the sample can be further analyzed through the CFG of the sample. Therefore, the present invention verifies whether the function of the sample is retained by comparing whether the CFG of the sample before and after modification has changed.
[0040] S2: Designing the observation space of intelligent agents based on feature engineering.
[0041] The complete features representing malware are divided into eight features according to their structure, namely, the sample's byte histogram, two-dimensional byte entropy histogram, string information, general file information, file header information, file chapter information, import table information, and export table information, and these features are optimized using feature engineering. The following parameters are defined:
[0042] x represents the original sample, which is converted into a vector v by the feature extractor f. x . F i Indicates v x The i-th feature among the eight features, v x [F i ][j] represents the j-th dimension sub-feature in the i-th feature, and v' x The present invention further analyzes the sample feature vector after modification by comparing the average distance change of each group of samples before and after modification. The average distance change (ADV) is calculated by the following formula:
[0043]
[0044] v x =f(x)
[0045] The features are effectively screened using the above formula, and the GBDT model is then used to determine the importance of different features in the original observation data to distinguish adversarial examples. By using the original state space to evade the detection of the EMBER detector, a subset of adversarial examples can be obtained. These samples are used together with the original samples to train the GBDT detector to distinguish adversarial examples. Figure 2 The importance of different features for distinguishing adversarial samples is shown in Figure 1. According to their different importance, the features such as byte histogram, byte entropy histogram and string information are considered as a whole. The general file information, file header information and chapter information account for a small proportion in the figure, and only the sub-features with higher importance are selected to be included in the new observation space. Through the above two methods, the present invention finally determines the observation space of the intelligent agent, such as Figure 3 as shown in .
[0046] S3: Improve the agent’s action space.
[0047] In previous work, there are a series of problems in the action space of intelligent agents. In order to solve these problems, a series of improvements have been made to the action space. The improved action space can be divided into four categories: injection, extension, modification and transformation.
[0048] The injection operation injects disturbances into different locations of the original sample. The injected disturbances come from specific content in the benign file. The extension operation involves moving specific content in the original sample backwards according to the structural characteristics and arrangement of the file, thereby creating space for modification. Two operations are introduced in the operation space, namely, the extension file header and the extension part. The modification operation involves changing certain features of the original sample without injecting new disturbances. There are five different modification behaviors for different locations and features of the sample, involving modification operations on the header, chapter table, and information at the end of the sample. The conversion operation refers to applying UPX technology to the original sample by packing or unpacking.
[0049] S4: Use the SAC (Soft-Actor-Critic) algorithm to train the agent to generate adversarial samples.
[0050] In this framework, the SAC (Soft-Actor-Critic) algorithm in reinforcement learning is used to train the agent to generate adversarial samples. First, an appropriate RL algorithm must be selected to train the agent to avoid the detector. The SAC algorithm is particularly suitable for tasks with large and complex state spaces because it contains the concept of information entropy. Traditional SAC algorithms are applicable to continuous action domains. However, in this task, the action space is discrete, so traditional SAC algorithms are not applicable. The present invention uses the SAC algorithm in the discrete domain to train the agent.
[0051] In addition, after the adversarial sample generation is completed, feature engineering and model interpretability are used to explain the generation of adversarial samples, thereby illustrating the weaknesses and shortcomings of currently popular malware detectors, such as Figure 4 and Figure 5 as shown in .
[0052] S5: Use policy distillation to train agents with better attack generalization against different types of detectors.
[0053] In view of the generalization of adversarial samples, this paper also proposes an adversarial sample generation framework based on multi-task strategy distillation, such as Figure 6 As shown in . The execution process of the proposed transferable attack framework is divided into two stages: training and testing. In the training stage, the multi-task policy distillation method is used to train the agent. In the data collection stage, a subset of the original samples is extracted from the training set, and multiple agents trained by the SAC algorithm are used as experts to interact with their respective detectors and produce outputs. These outputs, as well as the input and detector labels, are stored in the replay memory for subsequent updates. In the update stage, the student agent extracts experience from the replay buffer, generates corresponding outputs based on the input and label, calculates the distillation loss, and updates the policy network. Finally, multiple strategies from different tasks are merged into one strategy.
[0054] In the prediction phase, when the detector is known, the corresponding modification operation can be directly obtained by telling the agent the input and the detector label. When the detector is unknown, the agent initially generates all modification operations based on the sample state. Then, the modified samples generated by applying different operations are submitted to the unknown detector for query. If all modified samples fail to evade successfully, the sample with the lowest malicious score is selected as the input for the agent's next round of modification. According to the results of attack experiments against known detectors, the average number of queries used to generate adversarial samples does not exceed four times. Therefore, if the sample still cannot evade detection after four modifications, the detector label represented by the sample with the lowest score in each round is used to determine the potential detector that best matches the unknown detector. The specific method is to select the detector with the highest frequency among the four rounds of recorded labels. Subsequent modifications only use this label to obtain the corresponding output. Consistent with previous experiments, any modification that fails to evade detection within the maximum number of modifications or causes a change in the sample function is considered a failure. The attack effect of the framework is shown in the figure. Figure 7 as shown in .
[0055] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.
Claims
1. A reinforcement learning-based adversarial attack method to evade static malware detectors, characterized in that: The steps include: Step 1: Design an adversarial sample generation environment with a functional verification module; Step 2: Design the observation space of the intelligent agent based on feature engineering; Step 3: Improve the agent’s action space; Step 4: Use the Soft-Actor-Critic algorithm to train the agent to generate adversarial samples; Step 5: Use policy distillation to train an agent with stronger attack generalization to fight against different types of detectors.
2. The method for evading static malware detectors based on reinforcement learning according to claim 1, characterized in that: In step 1, the adversarial sample generation environment of the design with function verification module is specifically: the adversarial sample generation environment includes a file analyzer, a feature extractor, a static malware detector based on machine learning, a file modifier, and the original sample to be modified; this task has the following settings: The adversary's goal is to generate effective adversarial samples by performing a series of modification operations on the original samples while retaining their functionality to evade various detectors; during the attack, the adversary operates in a black box environment, and the adversary does not know the parameters, gradients, and other information of the victim model, nor does it know the original score of the sample; the adversary's goal is to generate functional adversarial malware in the problem space; if a sample successfully evades the detector through modification within the specified steps and passes the functional verification, it is regarded as a successful adversarial sample; conversely, if the number of modifications reaches the upper limit, or the sample structure is destroyed or the function is changed during the modification process, the modification is regarded as a failure.
3. The method for evading static malware detectors based on reinforcement learning according to claim 2, characterized in that: During the adversarial sample generation process, the file analyzer will analyze the original sample to evaluate the integrity, parsability, and maliciousness of the sample structure. If structural damage or parsing problems are detected during the analysis process, the sample will be considered damaged and excluded from subsequent modification operations; The feature extractor will then extract features from the analyzed samples to obtain features for the detector to assess the degree of maliciousness and for the agent to determine appropriate modification operations. File modifications will be made to the samples based on the corresponding modification actions output by the agent.
4. The method for evading static malware detectors based on reinforcement learning according to claim 3 is characterized in that: The file analyzer contains three modules, which analyze the readability, parsability and functional integrity of the sample respectively. First, the file analyzer uses Python's native file reading module to process the sample to ensure the extraction of byte content. After that, the file analyzer uses LIEF to parse the sample, extract data for feature construction, and perform subsequent file modifications; LIEF is a platform for parsing files and provides an API for accessing the internal data of files; finally, the file analyzer uses Angr to implement the verification operation of the function retention of the sample before and after modification; Angr is an open source binary analysis platform that can analyze binary files. It can recover the control flow graph (CFG) of binary files. The nodes of CFG represent basic instruction blocks, and the edges represent potential control flow transfers between them. It can determine the location and content of functions in binary files, and further analyze the functions of samples through the CFG of samples. Therefore, by comparing whether the CFG of samples before and after modification has changed, it can be verified whether the functions of samples have been preserved.
5. The method for evading static malware detectors based on reinforcement learning according to claim 1, characterized in that: In step 2, the observation space of the intelligent agent is designed based on feature engineering, specifically: the complete features representing the malware are divided into eight features according to their structure, namely, a byte histogram of the sample, a two-dimensional byte entropy histogram, string information, general information of the file, header information of the file, chapter information of the file, import table information, and export table information, and these features are optimized by feature engineering; Define the following parameters: x represents the original sample, which is converted into a vector v by the feature extractor f. x , F i Indicates v x The i-th feature among the eight features, v x [F i ][j] represents the j-th dimension sub-feature in the i-th feature, and v' x Represents the modified sample feature vector; further analysis is performed by comparing the average distance changes of each group of samples before and after modification. o(α) represents a very small value to prevent illegal division by zero. The average distance change ADV is calculated by the following formula: v x =f(x) The above formula is used to effectively screen the features, and then the GBDT model is used to determine the importance of different features in the original observation data to distinguish adversarial samples; a subset of adversarial examples is obtained by using the original state space to evade the detection of the EMBER detector; these samples are used together with the original samples to train the GBDT detector to distinguish adversarial examples; the observation space of the agent is finally determined through the above two methods.
6. The method for evading static malware detectors based on reinforcement learning according to claim 1, characterized in that: In step 3, the action space of the improved intelligent agent is specifically as follows: the action space is improved, and the improved action space is divided into four categories: injection, extension, modification and conversion; the interference is injected into different positions of the original sample through the injection operation; the injected disturbance comes from specific content in the benign file; the extension operation involves moving the specific content in the original sample backward according to the structural characteristics and arrangement of the file, thereby creating space for modification, and introducing two operations in the operation space, namely, extending the file header and the extension part; the modification operation involves changing certain features of the original sample without injecting new disturbances; for different positions and features of the sample, it involves modifying the header, chapter table and the information at the end of the sample; the conversion operation refers to applying UPX technology to the original sample by packing or unpacking.
7. The method for evading static malware detectors based on reinforcement learning according to claim 1, characterized in that: In step 4, the Soft-Actor-Critic algorithm is used to train the intelligent agent to generate adversarial samples. Specifically, the Soft-Actor-Critic algorithm includes the concept of information entropy and is suitable for tasks with large and complex state spaces. At the same time, after the adversarial sample generation is completed, feature engineering and the interpretability of the model are used to explain the generation of the adversarial sample, thereby explaining the weaknesses and deficiencies of the malware detector.
8. The method for evading static malware detectors based on reinforcement learning according to claim 1, characterized in that: In step 5, the strategy distillation training is used to attack an agent with stronger generalization to fight against different types of detectors, specifically: The proposed transferability attack framework is implemented in two phases: training and testing. In the training phase, the agent is trained by multi-task policy distillation. In the data collection phase, a subset of the original samples is extracted from the training set, and multiple agents trained by the SAC algorithm are used as experts to interact with their respective detectors and produce outputs. These outputs, as well as the input and detector labels, are stored in the replay memory for subsequent updates. In the update phase, the student agent extracts experience from the replay buffer, generates corresponding outputs based on the input and label, calculates the distillation loss, and updates the policy network. Finally, multiple policies from different tasks are merged into one policy. In the prediction stage, when the detector is known, the corresponding modification operation is directly obtained by telling the agent the input and the detector label. When the detector is unknown, the agent initially generates all modification operations based on the sample state; then, the modified samples generated by applying different operations are submitted to the unknown detector for query; if all modified samples fail to evade successfully, the sample with the lowest malicious score is selected as the input for the agent's next round of modification; according to the results of attack experiments against known detectors, the average number of queries used to generate adversarial samples does not exceed four times; therefore, if the sample still cannot evade detection after four modifications, the detector label represented by the sample with the lowest score in each round is used to determine the potential detector that best matches the unknown detector; the specific method is to select the detector with the highest frequency in the four rounds of recorded labels; subsequent modifications only use this label to obtain the corresponding output; consistent with previous experiments, failure to evade detection within the maximum number of modifications or any modification that causes a change in the sample's functionality is considered a failure.