Backdoor attack sample generation method and device of AI algorithm and related equipment

Abstract

The invention provides a backdoor attack sample generation method and device for an AI algorithm and related equipment, and the method comprises the steps: obtaining a sample program, carrying out the feature extraction of the sample program, and obtaining a feature set; performing confidence coefficient calculation on the feature set according to an SHAP model, and determining at least two feature regions in the feature set according to a confidence coefficient calculation result; initializing the at least two feature regions according to a set strategy to obtain initial back door features; on the basis of the feature set, optimizing the initial backdoor feature by using a genetic algorithm to obtain a target backdoor feature; and decompiling the target backdoor feature back to the sample program to generate an attack sample.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a method, device and related equipment for generating backdoor attack samples of an AI algorithm. Background Art

[0002] With the exponential growth of malware, methods such as code obfuscation and polymorphism have caused traditional malware detection methods to be unable to keep up with the pace of malware development. Security personnel have turned to machine learning and deep learning to address the challenge of large-scale malware detection. Machine learning-based malware classifiers can effectively combat traditional evasion methods and capture complex malicious features. Due to their fast detection speed, they usually serve as the first line of defense for malware detection together with rule-based methods.

[0003] However, the detection method based on machine learning has also attracted the attention of attackers, resulting in a large number of AI-related security issues. As a common means of network attack, backdoor attack has always been a key research direction in the field of network security. Nowadays, network attacks are complex and changeable. The feasibility of backdoor attacks in the field of Android malware has been confirmed by some studies. However, the current backdoor generation algorithm still has the disadvantages of not being applicable to real attack scenarios and not being able to guarantee both effectiveness and concealment at the same time, making it difficult for the generated samples to train the detection software and unable to achieve effective protection. Summary of the invention

[0004] In view of this, the present application proposes a method, device and related equipment for generating backdoor attack samples of an AI algorithm to solve or partially solve the above problems.

[0005] Based on the above purpose, this application provides a method for generating backdoor attack samples of an AI algorithm, including:

[0006] Acquire a sample program, perform feature extraction on the sample program, and obtain a feature set;

[0007] Performing confidence calculation on the feature set according to the SHAP model, and determining at least two feature regions in the feature set according to the confidence calculation result;

[0008] Initializing the at least two feature regions according to a set strategy to obtain an initial backdoor feature;

[0009] Based on the feature set, the initial backdoor feature is optimized using a genetic algorithm to obtain a target backdoor feature;

[0010] The target backdoor features are decompiled back into the sample program to generate an attack sample.

[0011] In some exemplary embodiments, extracting features from the sample program to obtain a feature set includes:

[0012] Reverse engineering the sample program using a compilation tool to obtain a first source code including a logic code of the sample program and a second source code including basic information of the sample program;

[0013] The first source code is digitized by one-hot encoding, and the second source code is digitized by counting the Markov transition probabilities of functions at package granularity and family granularity, so as to obtain the feature set.

[0014] In some exemplary embodiments, the calculating the confidence of the feature set according to the SHAP model includes:

[0015] Calculating the confidence of the feature set by using the SHAP model to obtain a confidence matrix of any feature in the feature set, taking the average of the confidence matrix to obtain an average confidence of any feature;

[0016] According to the average confidence of each feature, the average confidence matrix is ​​obtained.

[0017] In some exemplary embodiments, determining at least two feature areas in the feature set according to the confidence calculation result includes:

[0018] According to the average confidence matrix, selecting features with average confidence higher than a first threshold value to generate a first feature region;

[0019] According to the average confidence matrix, features with average confidence lower than a second threshold are selected to generate a second feature region.

[0020] In some exemplary embodiments, initializing the at least two feature areas according to a set strategy to obtain an initial backdoor feature includes:

[0021] For the first feature region, determine an average feature value of features in the first feature region, and perform initialization according to the average feature value;

[0022] For the second feature area, a random value under uniform distribution that satisfies a set feature value constraint condition in the feature set is determined, and initialization is performed according to the random value.

[0023] In some exemplary embodiments, the optimizing the initial backdoor features by using a genetic algorithm to obtain target backdoor features includes:

[0024] Selecting a set number of features from the feature set and inserting them into the initial backdoor features to obtain training samples, and repeatedly selecting the training samples to obtain a training set;

[0025] Retraining the training set using a genetic algorithm to determine fitness scores of samples in the training set;

[0026] Determine the training sample with the highest fitness score as the target backdoor feature.

[0027] In some exemplary embodiments, after generating the attack sample, the method further includes:

[0028] The attack samples are output to a collection platform for storage, so that an external malware detection system can be used for regular malware detection retraining.

[0029] Based on the same concept, the present application also provides a backdoor attack sample generation device of an AI algorithm, comprising:

[0030] The first module is used to obtain a sample program, perform feature extraction on the sample program, and obtain a feature set;

[0031] The second module is used to perform confidence calculation on the feature set according to the SHAP model, and determine at least two feature areas in the feature set according to the confidence calculation result;

[0032] A third module is used to initialize the at least two feature areas according to a set strategy to obtain an initial backdoor feature;

[0033] A fourth module is used to optimize the initial backdoor features by using a genetic algorithm based on the feature set to obtain target backdoor features;

[0034] The fifth module is used to decompile the target backdoor features back into the sample program to generate an attack sample.

[0035] Based on the same concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the program.

[0036] Based on the same concept, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement any of the methods described above.

[0037] As can be seen from the above, the present application provides a method, device and related equipment for generating backdoor attack samples of an AI algorithm, the method comprising: obtaining a sample program, extracting features from the sample program, and obtaining a feature set; performing confidence calculation on the feature set according to the SHAP model, and determining at least two feature areas in the feature set according to the confidence calculation result; initializing the at least two feature areas according to the set strategy to obtain an initial backdoor feature; optimizing the initial backdoor feature using a genetic algorithm on the basis of the feature set to obtain a target backdoor feature; decompiling the target backdoor feature back to the sample program to generate an attack sample. The present application uses the SHAP machine learning interpretability tool to select efficient features as the backdoor initial set, then designs a fitness function, and uses a genetic algorithm to perform backdoor iterative optimization under the guidance of the function. And the backdoor features can be converted into code lines and decompiled back to the original code, so that a backdoor attack can be carried out on the classifier in a real environment of data outsourcing. Experiments show that the backdoor attack samples proposed in this application are highly effective and concealed. The insertion algorithm not only ensures the normal operation of the software functions, but also effectively avoids the problems of pruning and redundant code, solving the problems that the current malware backdoor attack is not applicable to real scenarios and that it is difficult to balance effectiveness and concealment. Ultimately, it provides more effective experimental samples for detection software, which can effectively improve the performance of detection software. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flowchart of an exemplary method provided in an embodiment of the present application.

[0040] Figure 2 A schematic diagram of feature grouping according to an exemplary method provided in an embodiment of the present application.

[0041] Figure 3 A schematic diagram of the meaning of package granularity and family granularity provided in the embodiments of the present application.

[0042] Figure 4 A schematic diagram of the structure of an exemplary device provided in an embodiment of the present application.

[0043] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this specification more clear, this specification is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0045] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements, objects or method steps appearing before the word cover the elements, objects or method steps listed after the word and their equivalents, without excluding other elements, objects or method steps. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0046] As mentioned in the background technology section, machine learning-based detection methods have also attracted the attention of attackers, which has led to more targeted attack methods. For example, adversarial attacks occur in the inference phase, which make slight modifications to the sample's feature space so that the sample can bypass detection; poisoning attacks occur in the training phase. Since machine learning-based malware detectors have aging characteristics, they need to regularly collect a large number of samples from third-party platforms for retraining, which provides a natural poisoning point for attackers. By submitting poisoned samples to a third party, the poisoned samples can be downloaded to the training set by users. After retraining with the poisoned samples, the decision of the target model can be destroyed. As a variant of poisoning attacks, backdoor attacks construct backdoor triggers through careful calculations before poisoning. After the target model is trained with samples injected with backdoor triggers, it has a backdoor, causing it to correctly classify normal samples and misclassify samples with backdoors. These backdoors can be further divided into two categories: 1. Label tampering backdoor attack, that is, the original label of the sample is modified, 2. Clean label backdoor attack, that is, the original label of the sample is not changed. After generating the backdoor feature, the attacker will use the backdoor insertion algorithm to map the feature to the original code file.

[0047] Backdoor attacks, as a common means of network attack, have always been a key research direction in the field of network security. Nowadays, network attacks are complex and changeable. The feasibility of backdoor attacks in the field of Android malware has been confirmed by some studies. However, the backdoor generation algorithm still has the disadvantages of being not applicable to real attack scenarios and its effectiveness and concealment cannot be guaranteed at the same time. The backdoor insertion algorithm is easily pruned by static analyzers and may introduce redundant code.

[0048] In the process of researching samples of detection software, in order to carry out backdoor attacks on malware detection systems, current researchers have introduced some attack methods that have good attack effects in the field of image recognition into the field of malware detection systems. However, since software features are different from image features, they are highly complex and can easily damage the software itself during feature conversion, resulting in attack failure. Attack algorithms in the image field often find it difficult to achieve ideal results. There are problems such as the difficulty of implementing attacks in real scenarios and the difficulty of ensuring the effectiveness and concealment of attack algorithms at the same time.

[0049] In combination with the above actual situation, the embodiment of the present application provides a method for generating backdoor attack samples of an AI algorithm. The present application uses the SHAP machine learning interpretability tool to screen efficient features as the initial set of backdoors, then designs a fitness function, and uses a genetic algorithm to perform iterative optimization of the backdoor under the guidance of the function. And the backdoor features can be converted into lines of code and decompiled back to the original code, so that the classifier can be attacked by the backdoor in the real environment of data outsourcing. Experiments show that the backdoor attack samples proposed in this application are highly effective and concealed. The insertion algorithm not only ensures the normal operation of the software functions, but also effectively avoids the problems of pruning and redundant code, and solves the problem that the current malware backdoor attack real scene is not applicable, and the effectiveness and concealment are difficult to balance. Finally, a more effective experimental sample is provided for the detection software, which can effectively improve the performance of the detection software.

[0050] Figure 1 A flow chart of an exemplary method provided in an embodiment of the present application is shown.

[0051] like Figure 1 As shown, the method for generating backdoor attack samples of an AI algorithm exemplarily proposed in an embodiment of the present application includes the following steps.

[0052] Step 102: Obtain a sample program, perform feature extraction on the sample program, and obtain a feature set.

[0053] In this step, the sample program can be an application program in different systems. Since the corresponding systems are different, the feature extraction methods of various applications are also different. This difference is mainly reflected in the reverse engineering of the program itself, such as unpacking. There are corresponding reverse tools for different systems. In this embodiment, the Android platform is taken as an example, and the sample program here is the Android version of the software program.

[0054] After that, it is necessary to extract features from the sample program to obtain a feature set. In this process, in order to extract features, it is first necessary to disassemble the program, obtain the required features, and form these features into a feature set. Specifically, this step is mainly used to collect Android malware and benign software data used for backdoor generation, and preprocess the collected data. The preprocessing mainly includes three steps: decompilation, feature extraction using corresponding algorithms, and vectorization.

[0055] In some embodiments, for Android software, Apktool can be used to reverse engineer the packaged Android software. Here, Apktool is a powerful Android platform tool, which is mainly used to decompile and repackage APK files. It can decode the resources and Smali code in the APK, so that developers, reverse engineers, and people interested in the structure of Android applications can deeply understand the application behavior, reverse engineer, or modify the application to meet specific needs. Through Apktool reverse sample program, source code files such as class.dex and AndroidManifest.xml can be obtained, where class.dex contains the logic code of the app, and AndroidManifest.xml contains the basic information of the app, including the required permissions, etc. According to the different training feature sets used by the malware classifier, the modification of the original code file is divided into two categories: modifying the Androidmanifest.xml file and modifying the .smali source code file obtained by decompiling the .dex file.

[0056] After that, the features in the source code file can be digitized. For .xml files, the features can be divided into multiple groups according to the settings of the operator. The specific division can be set according to the specific scenario. Specifically, in one scenario, the features of the .xml file can be divided into eight groups, such as Figure 2As shown in the figure, these functions are extracted by classification and encoded by one-hot encoding. The value can indicate whether a specific permission / API / component exists. For example, for "permissions", if there are three permissions, "READ", "WRITE" and "EXECUTE", they are encoded as (1,0,0), (0,1,0) and (0,0,1) respectively. For the .smali file decompiled from the .dex file, the Markov transition probability of the function at the package granularity and family granularity is counted as the numerical feature of the .smali file. The meanings of package granularity and family granularity are as follows: Figure 3 That is, in some embodiments, the feature extraction of the sample program to obtain the feature set includes: using a compilation tool to reverse engineer the sample program to obtain a first source code (i.e., an .xml file) containing the logic code of the sample program, and a second source code (i.e., a .smali file) containing basic information of the sample program; performing numerical processing on the first source code by one-hot encoding (i.e., one-hot), and performing numerical processing on the second source code by statistically analyzing the Markov transition probability of functions at package granularity and family granularity, so as to obtain the feature set.

[0057] In more specific scenarios, the .xml feature uses one-hot encoding to convert character features in text data into numerical features, and the .dex feature is extracted by extracting the number of function calls. When the feature dimension is too high and SHAP cannot be calculated, SVM can be used to screen the feature importance and select the top 500 important features for SHAP calculation.

[0058] Step 104: Calculate the confidence of the feature set according to the SHAP model, and determine at least two feature regions in the feature set according to the confidence calculation result.

[0059] In this step, SHAP value (SHapley Additive exPlanations) is a method for explaining the predictions of machine learning models, which is based on the Shapley value theory in game theory. The core idea of ​​SHAP value is to calculate the marginal contribution of each feature to the model output, so as to explain the "black box model" at both the global and local levels. This method can be used for classification and regression models to generate graphical and quantitative explanation results to help users understand the decision-making process of the model.

[0060] In some embodiments, a set of efficient initial backdoor regions can be calculated based on SHAP. First, the SHAP value can be used to evaluate the impact of the model and feature set on the prediction results of each sample. The SHAP confidence matrix of each sample is calculated for the attack sample set, and its average is taken to form an average confidence matrix. In a binary classification problem, a higher confidence value indicates a greater contribution to the positive class, and a lower confidence value indicates a greater contribution to the negative class. This embodiment proposes two efficient initial backdoor region selection strategies, one is to select high average confidence features that have a greater impact on benign predictions, and the other is to select low average confidence features that have a smaller impact on the model to establish a connection between the backdoor trigger and the benign label. That is, the SHAP interpretability tool is used to calculate the average feature importance score from the preprocessed feature set (also referred to as the attack data set), and two strategies are used for initial backdoor region selection. Strategy one is to obtain a region with a higher score as a backdoor candidate position, and strategy two is to obtain a background region with a score close to 0 as a backdoor candidate position, so as to achieve the purpose of establishing an association relationship between the backdoor and the benign label. That is, in some embodiments, the confidence calculation of the feature set according to the SHAP model includes: performing confidence calculation on the feature set through the SHAP model to obtain a confidence matrix of any feature in the feature set, taking the average of the confidence matrix to obtain the average confidence of any feature; and obtaining an average confidence matrix according to the average confidence of each feature. Afterwards, determining at least two feature regions in the feature set according to the confidence calculation result includes: selecting features with an average confidence higher than a first threshold according to the average confidence matrix to generate a first feature region; selecting features with an average confidence lower than a second threshold according to the average confidence matrix to generate a second feature region. Among them, features higher than the first threshold are features with a higher average confidence, and features lower than the second threshold are features with a lower average confidence or 0. The specific settings of the first threshold and the second threshold can be determined according to the specific scenario.

[0061] Step 106, initializing the at least two feature regions according to the set strategy to obtain an initial backdoor feature.

[0062] In this step, after obtaining at least two feature regions in step 104, in order to facilitate subsequent selection thereof by a genetic algorithm, the two feature regions need to be initialized first.

[0063] In the above-mentioned embodiment, two strategies are mainly provided for the selection of feature regions, wherein strategy one is to select regions with a higher average confidence, which are more likely to be considered as benign features; and strategy two selects regions with a lower average confidence, which are more likely to be considered as background features. Furthermore, the initialization process for these two regions may also be different. Specifically, after the feature regions obtained using strategies one and two (here they can be called backdoor candidate regions, and the final location of the backdoor will be determined from the feature regions here), the features therein are initialized. For the features of strategy one, the initial feature value can be set to the average feature value of all benign samples under the feature, that is, the initial feature value of strategy one is the average feature value of benign samples. For the features of strategy two, the initial feature value is set to a random value under a uniform distribution that satisfies the feature value constraint, that is, strategy two is a uniformly distributed random value that satisfies the feature value constraint. That is, in some embodiments, the at least two feature regions are initialized according to the set strategy to obtain the initial backdoor feature, including: for the first feature region, determining the average feature value of the features in the first feature region, and initializing according to the average feature value; for the second feature region, determining a random value under a uniform distribution that satisfies the set feature value constraint in the feature set, and initializing according to the random value.

[0064] Step 108, based on the feature set, using a genetic algorithm to optimize the initial backdoor features to obtain target backdoor features.

[0065] In this step, the genetic algorithm (GA) uses a mathematical method and computer simulation to convert the problem-solving process into processes such as the crossover and mutation of chromosome genes in biological evolution. When solving more complex combinatorial optimization problems, it is usually possible to obtain better optimization results faster than some conventional optimization algorithms. Since there are generally multiple initial backdoor features, especially for the scenes in the aforementioned embodiments, the initial backdoor features can correspond to two feature areas, and these feature areas will contain multiple features, and then the genetic algorithm can be used for iterative retraining, and the fitness scores of the features of different combinations are determined by the genetic algorithm, and the different combinations are distinguished by the height of the fitness scores, and the desired combination is selected as the target backdoor feature. Specifically, the target backdoor feature can be selected with the highest fitness score.

[0066] In some embodiments, K benign software feature matrices can be selected from the feature set to insert the initial backdoor feature, generate samples labeled as benign, and add them to the training set to retrain the model. By calculating the fitness score, after N rounds of selection, crossover and mutation, until the fitness score of one of the iterations no longer increases, the algorithm is considered to have converged, and the backdoor feature with the highest fitness score can be determined as the target backdoor feature.

[0067] Specifically, the characteristics and characteristic values ​​of the individuals constitute a population of length L, which represents the backdoor characteristics. The algorithm adopts a fitness function T, as shown in the following formula.

[0068] T=a*S+b*V

[0069] Where S is the concealment score, S = count(F(X b )=original label),X b is the inserted backdoor feature, F() is the corresponding model; V is the validity score, V=count(F b (X b )=target label)+count(F b (X) = original label), original label represents the original label, target label represents the inserted label; a and b are the corresponding weights, a+b=1.

[0070] It can be seen that the fitness function T is the weighted average of the concealment score S and the effectiveness score V, which is used to evaluate the performance of backdoor samples under normal models and poisoned models. Among them, the concealment score S calculates the number of samples that retain the original label under the normal model, while the effectiveness score V evaluates the number of samples that the backdoor samples are transformed into the target category in the poisoned model and the recognition ability of normal samples in the backdoor model. By selecting backdoor triggers with higher fitness for crossover operations, the algorithm randomly exchanges features and ensures that the trigger length remains unchanged, and then performs random mutation to introduce perturbations. At the same time, the perturbation range is controlled according to the feature constraints of benign samples, thereby generating new backdoor triggers, and finally selecting the backdoor trigger with the highest fitness score. The backdoor trigger here is the target backdoor feature. That is, in some embodiments, the use of a genetic algorithm to optimize the initial backdoor feature to obtain a target backdoor feature includes: selecting a set number of features from the feature set and inserting them into the initial backdoor feature to obtain a training sample, and repeatedly selecting the training sample to obtain a training set; retraining the training set using a genetic algorithm to determine the fitness scores of the samples in the training set; and determining the training sample with the highest fitness score as the target backdoor feature.

[0071] In some other embodiments, the SVM algorithm may be used as a replacement model to calculate the intermediate fitness score.

[0072] Step 110, decompile the target backdoor feature back to the sample program to generate an attack sample.

[0073] In this step, after obtaining the target backdoor feature, it is necessary to map it to the sample program to complete the generation of the attack sample. Specifically, a benign sample can be obtained from the test sample set, and the sample can be decompiled using a decompilation tool such as apkTool, and the feature insertion method proposed above can be used to insert the feature into the decompiled code file, and then repackaged to obtain the attack sample.

[0074] In some embodiments, according to the foregoing, in step 102, the extracted source code files are mainly .xml files and .smali files. Furthermore, in this step, the features can be first mapped to the .xml file, and by declaring the "android.permission.TRANSMIT_IR" permission, the feature value of the corresponding position in the feature space can be changed from 0 (unused) to 1 (used). Correspondingly, if the feature value needs to be modified from 1 (used) to 0 (unused), it is only necessary to use a regular expression to match the relevant information and delete it, thereby achieving effective switching of the feature state. After that, continue to map to the .smali file, for API addition, it is defined as Inc(x, t, d), and a new method d is added in x of the .smali file to call APIt. In order to improve concealment, two obfuscation methods can be used: first, the actual call is prevented during dynamic execution through a try-catch statement; second, an empty function is defined and called through other classes. For API deletion, it is defined as Dec(x, t, d), and the function call position is located by regular matching, and reflection technology is used for obfuscation, so that the static analyzer cannot detect the deleted API calls.

[0075] Afterwards, after obtaining the attack sample, in some embodiments, it can be output, for example, a sample with a backdoor can be submitted to an open source intelligence collection platform, such as virusTotal, and the platform saves the sample and label to the database. Afterwards, the malware detection system needs to be retrained regularly to adapt to the latest malware detection, so it will use the API provided by intelligence collection platforms such as virusTotal to obtain training data. The training data contains the attack samples submitted by the attacker, so the malware system has a backdoor after retraining. Afterwards, the malware detection system has a backdoor, so the attacker can insert the backdoor into the malware and upload it to the target server. At this time, the malware detection system will detect it as benign, and because the insertion of the backdoor feature does not affect the function of the malware, the attacker can launch an attack at any time. In this way, the simulation degree of the backdoor attack sample can be improved, making it closer to the current actual backdoor program, thereby providing the malware detection system with samples that can be trained and analyzed, and providing a guarantee for it to detect the corresponding backdoor program. That is, according to the different features used by the classifier, the features to be mapped are divided into XML features and DEX features, and methods for deleting and adding features are defined. In order to prevent the backdoor code from being pruned by the static analyzer, code obfuscation and reflection are further adopted.

[0076] It can be seen from the above embodiments that the embodiments of the present application provide a method for generating backdoor attack samples of an AI algorithm, the method comprising: obtaining a sample program, extracting features from the sample program, and obtaining a feature set; performing confidence calculation on the feature set according to the SHAP model, and determining at least two feature areas in the feature set according to the confidence calculation result; initializing the at least two feature areas according to the set strategy to obtain an initial backdoor feature; optimizing the initial backdoor feature based on the feature set using a genetic algorithm to obtain a target backdoor feature; decompiling the target backdoor feature back to the sample program to generate an attack sample. This application uses the SHAP machine learning interpretability tool to screen efficient features as the backdoor initial set, then designs a fitness function, and uses a genetic algorithm to perform backdoor iterative optimization under the guidance of the function. And the backdoor features can be converted into code lines and decompiled back to the original code, so that a backdoor attack can be carried out on the classifier in a real environment of data outsourcing. Experiments show that the backdoor attack samples proposed in this application are highly effective and concealed. The insertion algorithm not only ensures the normal operation of the software functions, but also effectively avoids the problems of pruning and redundant code, solving the problems that the current malware backdoor attack is not applicable to real scenarios and that it is difficult to balance effectiveness and concealment. Ultimately, it provides more effective experimental samples for detection software, which can effectively improve the performance of detection software.

[0077] In a more specific scenario, it can be seen that the present application uses the SHAP model interpretable tool to extract features under two strategies to obtain the backdoor candidate area, then initializes the feature value of the backdoor candidate area and optimizes the backdoor trigger using a genetic algorithm to obtain the final backdoor trigger. An Android backdoor insertion program is designed to convert the trigger features from the feature space into real code lines, insert them into the original smali code file, and obtain the backdoor software after repackaging, and then provide samples to the malware detection platform. The backdoor generation algorithm of the present application includes a feature selection module and a genetic algorithm optimization module. The core technology of the feature selection module mainly includes three parts: initial backdoor selection strategy, fitness function design, and genetic algorithm optimization. Initial backdoor selection strategy: Using the SHAP machine learning interpretability tool, a set of efficient features are selected from the feature set as the initial backdoor. Fitness function design: Design a fitness function to evaluate the performance of the backdoor in view of the effectiveness and concealment of the backdoor. Genetic algorithm optimization: Under the guidance of the fitness function, the initial backdoor is iteratively optimized using a genetic algorithm to obtain a better backdoor. The backdoor insertion algorithm divides the features to be mapped into XML features and DEX features according to the feature category, and designs methods for adding and deleting features using code obfuscation and reflection technology according to different feature formats. Based on the above backdoor generation and backdoor insertion process, this application solves the shortcomings of the backdoor generation algorithm that it is not applicable to real attack scenarios and cannot guarantee effectiveness and concealment at the same time, and solves the problem that the backdoor insertion algorithm is easily pruned by the static analyzer and may introduce redundant code.

[0078] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of the embodiment of the present application can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0079] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0080] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a backdoor attack sample generation device of an AI algorithm.

[0081] refer to Figure 4 , the backdoor attack sample generating device of the AI ​​algorithm comprises:

[0082] The first module 410 is used to obtain a sample program, perform feature extraction on the sample program, and obtain a feature set.

[0083] The second module 420 is used to perform confidence calculation on the feature set according to the SHAP model, and determine at least two feature regions in the feature set according to the confidence calculation result.

[0084] The third module 430 is used to initialize the at least two feature areas according to the set strategy to obtain initial backdoor features.

[0085] The fourth module 440 is used to optimize the initial backdoor features by using a genetic algorithm based on the feature set to obtain target backdoor features.

[0086] The fifth module 450 is used to decompile the target backdoor feature back to the sample program to generate an attack sample.

[0087] In some exemplary embodiments, the first module 410 is further configured to:

[0088] Reverse engineering the sample program using a compilation tool to obtain a first source code including a logic code of the sample program and a second source code including basic information of the sample program;

[0089] The first source code is digitized by one-hot encoding, and the second source code is digitized by counting the Markov transition probabilities of functions at package granularity and family granularity, so as to obtain the feature set.

[0090] In some exemplary embodiments, the second module 420 is further configured to:

[0091] Calculating the confidence of the feature set by using the SHAP model to obtain a confidence matrix of any feature in the feature set, taking the average of the confidence matrix to obtain an average confidence of any feature;

[0092] According to the average confidence of each feature, the average confidence matrix is ​​obtained.

[0093] In some exemplary embodiments, the second module 420 is further configured to:

[0094] According to the average confidence matrix, selecting features with average confidence higher than a first threshold value to generate a first feature region;

[0095] According to the average confidence matrix, features with average confidence lower than a second threshold are selected to generate a second feature region.

[0096] In some exemplary embodiments, the third module 430 is further configured to:

[0097] For the first feature region, determine an average feature value of features in the first feature region, and perform initialization according to the average feature value;

[0098] For the second feature area, a random value under uniform distribution that satisfies a set feature value constraint condition in the feature set is determined, and initialization is performed according to the random value.

[0099] In some exemplary embodiments, the fourth module 440 is further configured to:

[0100] Selecting a set number of features from the feature set and inserting them into the initial backdoor features to obtain training samples, and repeatedly selecting the training samples to obtain a training set;

[0101] Retraining the training set using a genetic algorithm to determine fitness scores of samples in the training set;

[0102] Determine the training sample with the highest fitness score as the target backdoor feature.

[0103] In some exemplary embodiments, the fifth module 450 is further configured to:

[0104] The attack samples are output to a collection platform for storage, so that an external malware detection system can be used for regular malware detection retraining.

[0105] For the convenience of description, the above devices are described in terms of functions divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0106] The device of the above embodiment is used to implement the backdoor attack sample generation method of the corresponding AI algorithm in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0107] Based on the same concept, corresponding to any of the above-mentioned embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a backdoor attack sample generation method of the AI ​​algorithm as described in any of the above embodiments is implemented.

[0108] Figure 5A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0109] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0110] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0111] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0112] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0113] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0114] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0115] The electronic device of the above embodiment is used to implement the backdoor attack sample generation method of the corresponding AI algorithm in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0116] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the backdoor attack sample generation method of the AI ​​algorithm described in any of the above embodiments.

[0117] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0118] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the backdoor attack sample generation method of the AI ​​algorithm as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0119] Based on the same concept, corresponding to any of the above-mentioned embodiments, the present application also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the method for generating backdoor attack samples of the AI ​​algorithm. Corresponding to the execution subject corresponding to each step in each embodiment of the method for generating backdoor attack samples of the AI ​​algorithm, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0120] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the backdoor attack sample generation method of the AI ​​algorithm as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0121] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0122] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0123] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0124] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for generating backdoor attack samples of an AI algorithm, characterized in that: include: Acquire a sample program, perform feature extraction on the sample program, and obtain a feature set; Performing confidence calculation on the feature set according to the SHAP model, and determining at least two feature regions in the feature set according to the confidence calculation result; Initializing the at least two feature regions according to a set strategy to obtain an initial backdoor feature; Based on the feature set, the initial backdoor feature is optimized using a genetic algorithm to obtain a target backdoor feature; The target backdoor features are decompiled back into the sample program to generate an attack sample.

2. The method according to claim 1, characterized in that The step of extracting features from the sample program to obtain a feature set includes: Reverse engineering the sample program using a compilation tool to obtain a first source code including a logic code of the sample program and a second source code including basic information of the sample program; The first source code is digitized by one-hot encoding, and the second source code is digitized by counting the Markov transition probabilities of functions at package granularity and family granularity, so as to obtain the feature set.

3. The method according to claim 1, characterized in that The calculating the confidence of the feature set according to the SHAP model includes: Calculating the confidence of the feature set by using the SHAP model to obtain a confidence matrix of any feature in the feature set, taking the average of the confidence matrix to obtain an average confidence of any feature; According to the average confidence of each feature, the average confidence matrix is ​​obtained.

4. The method according to claim 3, characterized in that The determining of at least two feature areas in the feature set according to the confidence calculation result includes: According to the average confidence matrix, selecting features with average confidence higher than a first threshold value to generate a first feature region; According to the average confidence matrix, features with average confidence lower than a second threshold are selected to generate a second feature region.

5. The method according to claim 4, characterized in that Initializing the at least two feature areas according to the set strategy to obtain the initial backdoor feature includes: For the first feature region, determine an average feature value of features in the first feature region, and perform initialization according to the average feature value; For the second feature area, a random value under uniform distribution that satisfies a set feature value constraint condition in the feature set is determined, and initialization is performed according to the random value.

6. The method according to claim 1, characterized in that The method of optimizing the initial backdoor features by using a genetic algorithm to obtain target backdoor features includes: Selecting a set number of features from the feature set and inserting them into the initial backdoor features to obtain training samples, and repeatedly selecting the training samples to obtain a training set; Retraining the training set using a genetic algorithm to determine fitness scores of samples in the training set; Determine the training sample with the highest fitness score as the target backdoor feature.

7. The method according to claim 1, characterized in that After generating the attack sample, the method further includes: The attack samples are output to a collection platform for storage, so that an external malware detection system can be used for regular malware detection retraining.

8. A backdoor attack sample generation device for an AI algorithm, characterized in that: include: The first module is used to obtain a sample program, perform feature extraction on the sample program, and obtain a feature set; The second module is used to perform confidence calculation on the feature set according to the SHAP model, and determine at least two feature areas in the feature set according to the confidence calculation result; A third module is used to initialize the at least two feature areas according to a set strategy to obtain an initial backdoor feature; A fourth module is used to optimize the initial backdoor features by using a genetic algorithm based on the feature set to obtain target backdoor features; The fifth module is used to decompile the target backdoor features back into the sample program to generate an attack sample.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to implement the method according to any one of claims 1 to 7.

Images