Techniques for training deep neural networks

Abstract

Techniques for training deep neural networks are disclosed. For example, various embodiments generally relate to techniques for training deep neural networks, such as using an iterative approach. Some embodiments particularly relate to a deep neural network (DNN) training system that generates a reinforced DNN by iteratively training the DNN with images that were misclassified by previous iterations of the DNN. For example, one or more embodiments may include logic to generate adversarial images that were misclassified by a first DNN previously trained with a set of sample images. In some embodiments, the logic may determine a second training set that includes adversarial images that were misclassified by the first DNN and the first training set of one or more sample images. The second training set may be used to train a second DNN. In various embodiments, the above process may be repeated for a predetermined number of iterations to produce the reinforced DNN.

Description

Background Art

[0001] Machine learning involves studying and building algorithms that can learn from data and make predictions about the data. Deep neural networks can implement algorithms to perform a type of machine learning called deep learning. Typically, deep learning can utilize a cascade of many layers of artificial neurons (such as nonlinear processing units). Typically, each successive layer uses the output of the previous layer as input. Overall, artificial neurons can utilize deep learning algorithms to perform feature extraction and transformation. Deep learning can include supervised algorithms and unsupervised algorithms. Generally, unsupervised algorithms are used for pattern analysis, while supervised algorithms are used for pattern classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Figure 1 An embodiment of a first operating environment is shown.

[0003] Figure 2A An embodiment of a second operating environment is shown.

[0004] Figure 2B An embodiment of a third operating environment is shown.

[0005] Figure 3A An embodiment of a fourth operating environment is shown.

[0006] Figure 3B An embodiment of a fifth operating environment is shown.

[0007] Figure 4 An embodiment of a sixth operating environment is shown.

[0008] Figure 5 An embodiment of a first logic flow is shown.

[0009] Figure 6 An embodiment of a second logic flow is shown.

[0010] Figure 7 An embodiment of a storage medium is shown.

[0011] Figure 8 An embodiment of a computing architecture is shown.

[0012] Figure 9 An embodiment of a communication architecture is shown. DETAILED DESCRIPTION

[0013] For example, various embodiments generally relate to techniques for training deep neural networks, such as using an iterative approach. Some embodiments particularly relate to a deep neural network (DNN) training system that generates a reinforced DNN by iteratively training the DNN with images misclassified by previous iterations of the DNN. In one or more embodiments, for example, a device may include memory and logic, at least a portion of which is implemented in a circuit coupled to the memory. In one or more such embodiments, the logic may generate adversarial images that were misclassified by a first DNN previously trained with a set of sample images. In some embodiments, the logic may determine a second training set that includes adversarial images misclassified by the first DNN and the first training set of one or more sample images. In some such embodiments, the second training set may be used to train a second DNN. In various embodiments, the above process may be repeated for a predetermined number of iterations to produce a reinforced DNN. These and other embodiments are described and claimed.

[0014] Some challenges faced by DNNs include the inability to properly classify images in a reliable and efficient manner. These challenges may arise from the DNN's inability to properly identify images that have only a few pixels changed. For example, an adversary may be able to craft images that the DNN cannot properly classify, known as adversarial images. In such instances, such adversarial images may contain street signs that the DNN would misclassify, leading to accidents. Furthermore, seemingly meaningless images may be misclassified by the DNN. For example, a DNN trained for image recognition may classify an image that appears to be white noise as a cheetah. Further complicating matters, in various embodiments, DNNs may be used to detect malware. In some instances, an adversary may be able to avoid detection by the DNN by changing minor or insignificant portions of the malware code. These and other factors may result in DNNs with poor performance and limited applicability. Such limitations may significantly reduce the power, usability, and reliability of DNNs, resulting in inefficient systems with limited benefits.

[0015] Various embodiments described herein include a DNN training system that generates a reinforced DNN through an iterative training process. In some embodiments, malware can be viewed as images generated from bytecode. In some such embodiments, these images can be used to train a DNN to perform malware classification. In various embodiments, the DNN training system can generate adversarial images that are misclassified by the DNN trained to perform malware classification. In various such embodiments, the adversarial images can then be used to train future DNN iterations. In this way, improved classification accuracy, particularly for adversarial images, can be achieved through the reinforced DNN, resulting in a more robust DNN. In one or more embodiments, image analysis for malware detection can reveal undetected relationships. For example, malware images can show similarities within a family as well as differences across families. In some embodiments, feature extraction can be avoided by directly converting bytecode into pixel intensities for classification. In some such embodiments, avoiding feature extraction in this way can improve efficiency while achieving high accuracy. In these and other ways, DNN training systems can achieve improved and more robust image classification, such as in malware detection, to achieve improved DNNs with greater reliability, better flexibility, enhanced capabilities, and higher efficiency, resulting in several technical effects and advantages.

[0016] According to the general reference to the notation and nomenclature used in this article, one or more parts of the following specific embodiments can be presented in terms of the program process performed on a computer or computer network. These procedural descriptions and representations are used by those skilled in the art to most efficiently pass the essence of their work to other persons skilled in the art. Process is generally understood to be a self-consistent sequence of operations that leads to a desired result. These operations are those operations that require physical manipulation of physical quantities. Usually, but not necessarily, these quantities take the form of electrical, magnetic or optical signals that can be stored, transmitted, combined, compared and otherwise manipulated. Mainly for the reason of common usage, it has proven to be convenient to refer to these signals as bits, values, elements, symbols, characters, items, numbers, etc. from time to time. However, it should be noted that all of these and similar terms are intended to be associated with appropriate physical quantities and are merely convenient labels applied to those quantities.

[0017] Furthermore, these manipulations are often referred to in terms of addition or comparison, which are often associated with mental operations performed by a human operator. However, in any of the operations described herein that form part of one or more embodiments, in most cases, no such capabilities of a human operator are necessary or required. Instead, these operations are machine operations. Useful machines for performing the various operations of the embodiments include general-purpose digital computers that are selectively activated or configured by a computer program stored therein and written in accordance with the teachings of this document; and / or include devices specifically constructed for the desired purpose. The embodiments also relate to devices or systems for performing these operations. These devices may be specially constructed for the desired purpose, or they may comprise general-purpose computers. The required structure of each such machine will be apparent from the given description.

[0018] Reference will now be made to the accompanying drawings, in which the same reference numerals are used throughout to refer to similar elements. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of the description. However, it will be apparent that each novel embodiment may be implemented without these specific details. In other examples, a plurality of well-known structures and devices are shown in block diagram form to facilitate the description of these structures and devices. The present invention is intended to encompass all modifications, equivalents, and alternatives within the scope of the claims.

[0019] Figure 1 An example of an operating environment 100 is shown that can represent various embodiments. The operating environment 100 can include a deep neural network (DNN) training system 102. In the operating environment 100, the DNN training system 102 can include a training set manager 104, a DNN trainer 106, and an adversarial image generator 108. In one or more embodiments described herein, the DNN training system 102 can receive input 101 and generate an enhanced DNN 150 based on the input. The embodiments are not limited in this context.

[0020] In various embodiments, the components of the DNN training system 102 may interoperate to produce a hardened DNN 150. In some embodiments, creating the hardened DNN 150 may include generating adversarial examples (e.g., images) to attack a deep learning algorithm for malware detection and / or performing iterative training on a sequenced DNN using the adversarial examples. For example, the training set manager 104 may determine a first training set based on the input 101, and the DNN trainer 106 may then train the first DNN to classify images using the first training set. In some such examples, the adversarial image generator 108 may generate one or more adversarial images that are misclassified by the first DNN. In one or more embodiments, the one or more adversarial images may be generated using an analytical attack implemented by the adversarial image generator 108. As will be described in more detail below, in one or more such embodiments, the analytical attack implemented by the adversarial image generator 108 may include causative attacks and evolutionary attacks.

[0021] In some embodiments, one or more adversarial images misclassified by the first DNN may then be provided to the training set manager 104 for inclusion in the second training set. In various embodiments, the process of generating adversarial images, determining an updated training set, and training the DNN with the updated training set may be repeated so that an iteration parameter is satisfied. For example, the input 101 may include an iteration parameter of five. In such an example, this may correspond to the DNN training system 102 performing five rounds of the following operations: generating adversarial images, determining an updated training set, and training the DNN with the updated training set. In one or more embodiments, the DNN trained with the training set including the last round of adversarial images generated may be output as the enhanced DNN 150.

[0022] Figure 2A An example of an operating environment 200A is shown that can represent various embodiments. Operating environment 200A can include training set manager 104 and DNN trainer 106. In operating environment 200A, training set manager 104 can receive iteration parameters 201 and initial training data set 202 as input 101. In various embodiments described herein, training set manager 104 can convert initial training data set 202 into training set 216-1. In various such embodiments, training set 216-1 can be passed to the DNN trainer and used to train the DNN. In some embodiments, iteration parameters 201 can determine how many different training sets are created for training the DNN. The embodiments are not limited in this context.

[0023] In one or more embodiments, initial training dataset 202 may include one or more segments of training data 204-1, 204-2, 204-n (i.e., training data 204). In one or more such embodiments, some or all segments of training data 204 in initial training dataset 202 may include assembly files (e.g., assembly files 206-1, 206-2, 206-n, or assembly file 206) containing bytecodes (e.g., bytecodes 208-1, 208-2, 208-n, or bytecode 208). In various embodiments, each segment of training data may be associated with a category (e.g., category 210-1, 210-2, 210-n, or category 210). In various such embodiments, the association of categories can be used in a deep learning algorithm to train a deep neural network (DNN). For example, a DNN may be trained to detect malware. In such an example, category 210 may include "malware" and "non-malware."

[0024] In various embodiments, the training set manager 104 may convert the initial training data set 202 into a training set 216-1. In various such embodiments, the training data 204-1, 204-2, 204-n may be converted into sample images 212-1, 212-2, 212-n, respectively (i.e., sample images 212). In some embodiments, the training set manager 104 may convert the training data into sample images by directly converting the associated bytecodes into pixel intensities. In one or more embodiments, the sample images 212-1, 212-2, 212-n may include categories 214-1, 214-2, 214-n, respectively (i.e., categories 214). In one or more such embodiments, the categories 214 may be the same as or similar to the categories corresponding to the bytecodes used to create the sample images (e.g., categories 210-1, 210-2, 210-n). In various embodiments, the categories 210 may be a subset of the categories 214. In other embodiments, the categories 214 may be a subset of the categories 210.

[0025] In some embodiments, the training set manager 104 may receive an initial training dataset 202 and iteration parameters 201 as input 101. In various embodiments, the initial training dataset 202 may include uncontaminated data samples having K categories. In one or more embodiments, the iteration parameters 201 may include a 1-bit parameter M for specifying the number of iterations for strengthening the DNN. Once the initial training dataset 202 is converted into a training set 216-1, the training set 216-1 may be provided to the DNN trainer 106 for training the first round of the DNN.

[0026] Figure 2BAn example of an operating environment 200B is shown that can represent various embodiments. Operating environment 200B may include training set manager 104, DNN trainer 106, and adversarial image generator 108. In operating environment 200B, training set manager 104 may determine training set 216-2 based on training set 216-1 and adversarial image set 254-1. Thus, in operating environment 200B, adversarial image set 254-1 was previously generated, such as by adversarial image generator 108, based on an analysis attack performed on a DNN trained with training set 216-1. The embodiments are not limited in this context.

[0027] In various embodiments, the training set manager 104 may receive an adversarial image set 254-1 from the adversarial image generator 108. In one or more embodiments, the adversarial images 252-1, 252-2, 252-n may each be associated with a category 253-1, 253-2, 253-n (i.e., category 253). In one or more such embodiments, each of the categories 253 may be different from category 210. Thus, in various embodiments, the training set 216-2 may include K+1 categories, as opposed to the K categories of the training set 216-1. For example, category 210 may include "malware" or "non-malware," while category 253 may include "induced adversarial images" and "evolved adversarial images." In some embodiments, the categories 253 may all be the same category, such as "adversarial images."

[0028] In some embodiments, the training set manager 104 may determine a training set 216-2 based on the training set 216-1 and the adversarial image set 254-1. In one or more embodiments described herein, the training set manager 104 may include one or more sample images 212 of the training set 216-1 and one or more adversarial images 252-1, 252-2, 252-n (i.e., adversarial images 252) of the adversarial image set 254-1 in the training set 216-2. In various embodiments, the training set 216-2 may include each sample image 212 of the training set 216-1 and each adversarial image 252 of the adversarial image set 254-1. In some embodiments, the training set manager 104 may then provide the training set 216-2 to the DNN trainer 106 to train the second DNN.

[0029] Figure 3AAn example of an operating environment 300A is shown that can represent various embodiments. Operating environment 300A can include training set manager 104, DNN trainer 106, and adversarial image generator 108. In operating environment 200A, DNN trainer 106 can receive training set 216-1 from training set manager 104. In the illustrated embodiment, DNN trainer 106 can include trainer set 216-1, untrained DNN 302, trained DNN 304-1, and iteration tracker 306. In various embodiments described herein, DNN trainer 106 can use training set 216-1 on untrained DNN 302 to generate trained DNN 304-1. In various such embodiments, untrained DNN 302 can include a deep learning algorithm to which training set 216-1 is provided as input. In some such embodiments, based on training set 216-1, the deep learning algorithm can generate trained DNN 304-1. The embodiments are not limited in this context.

[0030] In some embodiments, the DNN trainer 106 may receive the training set 216-1 from the training set manager 104. In some such embodiments, the training set 216-1 may be applied to the untrained DNN 302 to generate the trained DNN 304-1. In various embodiments, the iteration tracker 306 may track the number of iterations that the DNN training system 102 has performed. In various such embodiments, the iteration tracker 306 may determine the desired number of iterations based on the iteration parameter 201. In the operating environment 300A, the iteration tracker 306 may determine that more iterations are to be performed. For example, the iteration tracker 306 may include a counter that is initially set to match the iteration parameter 201. In such instances, the counter may be decremented each time a trained DNN is generated. In some such instances, if the counter is above zero, the trained DNN may be provided to the adversarial image generator 108. In other such instances, if the counter value is zero, the trained DNN may be output as the reinforced DNN 150.

[0031] In various embodiments, the trained DNN 304-1 can be a convolutional neural network capable of classifying malware represented as images generated from corresponding bytecode. In various such embodiments, resizing the width and / or height of the image may not significantly affect classification accuracy. In some embodiments, the trained DNN 304-1 can have a multi-layer perceptron architecture. In one or more embodiments, the trained DNN 304-1 can perform a two-category classification task.

[0032] Figure 3BAn example of an operating environment 300B that can represent various embodiments is shown. Operating environment 300B can include training set manager 104 and DNN trainer 106. In operating environment 300B, DNN trainer 106 can utilize training set 216-2 received from training set manager 104 to generate a trained DNN 304-2. In the illustrated embodiment, trained DNN 304-2 is output as a reinforced DNN 150. In such embodiments, in response to iteration tracker 306 determining that an appropriate number of iterations have been performed, trained DNN 304-2 can be output as a reinforced DNN 150. Thus, in operating environment 200B, iteration parameter 201 is set to 2. However, it will be understood that in other embodiments, iteration parameter 201 can be set to any positive integer. The embodiments are not limited in this context.

[0033] Figure 4 An example of an operating environment 400 is shown that can represent various embodiments. The operating environment 400 can include the training set manager 104, the DNN trainer 106, and the adversarial image generator 108. In the operating environment 400, the adversarial image generator 108 can include an induced image generator 402 and an evolved image generator 404. In the illustrated embodiment, the adversarial image generator 108 can receive the training set 216-1 from the training set manager 104 and the trained DNN 304-1 from the DNN trainer 106. In various embodiments described herein, the induced image generator 402 and the evolved image generator 404 can perform an analysis attack on the trained DNN 304-1 to generate the adversarial image set 254-1. It will be appreciated that the adversarial image generator 108 generates the adversarial image set at any iteration of the trained DNN. For example, the training set 216 - 2 and the trained DNN 304 - 2 may be utilized by the induced image generator 402 and the evolved image generator 404 to generate the adversarial image set 254 - 2 (not shown). The embodiments are not limited in this context.

[0034] In one or more embodiments, induced image generator 402 may perform an induced analysis attack on trained DNN 304-1 to generate one or more adversarial samples or images in set 254-1, and / or evolved image generator 404 may perform an evolved analysis attack on trained DNN 304-1 to generate one or more adversarial images in set 254-1. Generally, an analysis attack may include, for example, altering one or more portions of a base image (e.g., sample image 212-n) to generate a test image; providing the test image to a trained DNN (e.g., trained DNN 304-1) for classification; and, when the trained DNN classifies the test image as different from a target classification or category, including the test image as an adversarial image (e.g., adversarial image 252-2) in adversarial image set 254-1. In some embodiments, the target classification may be the category of the base image. In one or more embodiments, adversarial images or samples may fall into two general categories. In one or more such embodiments, a first category may include malware samples that appear very similar to clean samples, and a second category may include clean samples that appear very similar to malware samples. In various embodiments, the first category may pass undetected by the unenhanced DNN, while the second category may increase false positives.

[0035] In some embodiments, the induced image generator 402 may perform an induced attack as follows. Given a valid input x (e.g., sample image 212-1), find a similar x' such that the classification output C(x)≠C(x'), but x and x' are close according to some distance metric. In various embodiments, the induced attack attempts to change the minimum number of pixels (such as in sample image 212-2) that causes the trained DNN to classify x' differently than it would classify x. In one or more embodiments, the induced attack may be implemented via a fast gradient sign method. In one or more such embodiments, the fast gradient sign method may be defined as In some embodiments, the fast gradient sign method may enable fast and efficient generation of adversarial images.In various embodiments, induced attacks (such as those performed by induced image generator 402) may produce adversarial images that are classified as benign images.

[0036] In various embodiments, the evolutionary image generator 404 can perform an evolutionary attack as follows. Consider a set of sample images (e.g., sample image 212) as a population. Given a valid input x, use an evolutionary algorithm to find a dissimilar x' such that x' is white noise but is classified as a valid label by the trained DNN. In some embodiments, in each iteration, the evolutionary algorithm can select a random organism (e.g., sample image 212-n), randomly mutate it, and replace it based on optimizing a fitness function. In one or more embodiments, the evolutionary attack can be implemented using a multi-dimensional archive of phenotypic elites (MAP-elite). In one or more such embodiments, MAP-elite can implement simultaneous evolution of a population containing individuals with good scores.

[0037] In one or more embodiments, analyzing the attack and / or adversarial image can serve as a guide to changing or injecting code in an assembly file (e.g., assembly file 206-2). In one or more such embodiments, changing or injecting code in the assembly file can be directed to avoiding classification as malware by a trained DNN. In one or more such embodiments, a sliding window technique on an image can be used to approximate locations in the assembly file. In various embodiments, the sliding window technique can be used on a base image (e.g., a sample image on which the misclassified test image is based). In various such embodiments, the perturbed or changed portion can be a patch identified by the sliding window technique. In some embodiments, this can be used to determine which portion of the bytecode has the heaviest weight when involving a trained DNN in making predictions.

[0038] In various embodiments, if the patch is larger, then the others may include locations for code injection in the corresponding assembly file. In one or more embodiments, the locations identified in the base image by the sliding window technique are pixels that cannot be accurately mapped back to the assembly file (which is text). Therefore, in one or more such embodiments, the portion identified by the sliding window technique approximates the portion of the assembly file that has bytecode that needs to be changed. In some embodiments, symbolic execution may be used to evaluate the effectiveness of the code modification / injection. In some such embodiments, when adversarial images are included for classification, the accuracy of the deep learning algorithm may be reduced and may include a false positive rate.

[0039] In some embodiments, the DNN training system 102 can be directed to enhancing deep learning resiliency against analytical attacks. Thus, as a defense mechanism, the DNN training system 102 can rebuild the iterative layers of the DNN and / or utilize adversarial images to enhance the robustness of the deep learning algorithm. In one or more embodiments, the operation of the DNN training system 102 can proceed as follows. The input to the first DNN "DNN1" (e.g., input 101) can be an uncontaminated data sample or image with K categories and a 1-bit parameter M that specifies the number of iterations used to strengthen the DNN (e.g., iteration parameter 201). DNN1 can be trained on the uncontaminated data sample. The DNN training system 102 can then generate adversarial examples, such as examples constructed from induced attacks and evolved attacks. These attacks can slightly modify the uncontaminated data sample (e.g., base image) to produce a test image. When DNN1 classifies a test image into the wrong category (e.g., not the target category), the test image can be identified as an adversarial image and included in an adversarial image set (e.g., adversarial image set 254-1). In some embodiments, the adversarial image can be added to a subsequent training set (e.g., training set 216-2) as a new category. In such embodiments, training set 216-2 will have K+1 categories.

[0040] In one or more embodiments, a second DNN "DNN2" can be trained on the training set 216-2. In one or more such embodiments, the induced image generator 402 and the evolved image generator 404 can then perform an analysis attack on DNN2. In various embodiments, the DNN training system 102 can repeat this process until the number of iterations M is reached. For example, in malware analysis, the input may be a malware image and an iteration parameter of 8. In such an example, DNN1 can be trained on the set of malware images. In some embodiments, a 2-layer perceptron (2-MLP) can be utilized. In some such embodiments, the 2-MLP can be utilized based on speed considerations.

[0041] In some embodiments, different architectures for the DNN can be utilized depending on the data size. For example, for an MLP, the number of layers may vary. In another example, for LeNet, the filter size may vary. In one or more embodiments, the DNN training system 102 can then use induced and evolved attacks to launch an adversarial attack to identify the adversarial image. In various embodiments, the induced attack can be implemented based on minimizing the L-infinity or Chebyshev distance of the adversarial image. In some embodiments, the perturbed or altered portion can serve as a guide for altering the associated bytecode. In some such embodiments, the location of the bytecode in the assembly file can be approximated using a sliding window technique performed on the image. In one or more embodiments, symbolic execution can be used to evaluate effectiveness. In various embodiments, after verification, a new training set is created that includes the adversarial image, and the resulting training set thus has K+1 categories. In various such embodiments, DNN 2 can be trained on the resulting training set with K+1 categories, and this process can be repeated until DNN 8 is trained. In some embodiments, DNN 8 can be output as the reinforced DNN 150. In one or more embodiments, classification of adversarial images may improve within five iterations.

[0042] In various embodiments, the DNN training system 102 can protect against various analytical attacks on deep learning. In some embodiments, the DNN training system 102 can be adapted to improve the resilience of deep learning used in one or more of autonomous driving, computer vision, speech recognition, the Internet of Things, and malware detection. For example, for autonomous driving, when an autonomous vehicle's camera captures a real-world image, but an attack perturbs a minimal number of pixels in the image, the unenhanced machine learning algorithm used by the vehicle may misclassify the image. In such examples, this misclassification may lead to an accident. Therefore, in one or more embodiments, the DNN training system 102 can prevent dangerous consequences caused by misclassification. In various embodiments, overcoming adversarial images with the DNN training system 102 can avoid safety issues and / or give deep learning algorithms a more accurate understanding of the tasks they solve.

[0043] Figure 5 One embodiment of a logic flow 500 is shown, which may represent operations that may be performed in various embodiments, such as in conjunction with an embodiment for generating an enhanced DNN 150. The logic flow 500 may represent the operations performed by Figure 1-4 Some or all of the operations performed by one or more components of the operating environments 100, 200, 300, 400, such as the DNN training system 102, the training set manager 104, the DNN training 106, or the adversarial image generator 108. The embodiments are not limited in this context.

[0044] exist Figure 5 In the illustrative embodiment shown in FIG, logic flow 500 may begin at block 502. At block 502, "Input iteration parameters," iteration parameters may be received as input. For example, training set manager 104 may receive iteration parameters 201 in input 101. Proceeding to block 504, "Input samples," samples may be received as input. For example, training set manager 104 may receive initial training data set 202 as input samples in input 101.

[0045] At block 506, "Determine Current Training Set," a current training set may be determined. For example, training set 216-1 may be determined from initial training data set 202. In various embodiments, training set 216-1 may include one or more sample images. In some embodiments, training set manager 104 may convert bytecodes to pixel intensities to generate sample images. For example, bytecode 208-1 may be converted to pixel intensities to generate sample image 212-1.

[0046] Continuing to block 508, “Train DNN-i on Current Training Set,” the current iteration of the DNN can be trained on the current training set. For example, trained DNN 304-1 can be generated based on training set 216-1. In some embodiments, the current training set can be input to an untrained DNN (e.g., untrained DNN 302) to generate the current trained DNN (e.g., trained DNN 304-1).

[0047] At block 510, "Enough iterations performed?", a determination may be made as to whether a sufficient number of iterations have been performed. For example, iteration tracker 306 may count the number of iterations performed and compare it to iteration parameter 201. If sufficient iterations have been performed, the current DNN may be output as the enhanced DNN 150. On the other hand, if insufficient iterations have been performed, the iteration count may be incremented at block 512, and logic flow 500 may proceed to block 514.

[0048] At block 514, "Generate Test Image," a test image can be generated. For example, one or more portions of the sample image can be altered to generate the test image. Proceeding to block 515, "Provide Test Image to DNN for Classification," the test image can be provided to the DNN for classification. Continuing to block 516, "Is Test Image Appropriately Classified?", a determination can be made as to whether the test image was appropriately classified by the DNN. For example, if the classification of the test image matches the target classification, then the test image was appropriately classified. However, if the classification of the test image does not match the target classification, then the test image was inappropriately classified.

[0049] When the test image is properly classified, logic flow 500 may proceed to block 518. At block 518, "Change Portion of Test Image," the properly classified portion of the test image is changed, and at block 515, the changed test image is resubmitted to the DNN for classification. Referring back to block 516, when the test image is not properly classified, at block 520, the test image may be identified as an adversarial image. Continuing to block 522, "Enough adversarial images identified?", a determination may be made as to whether sufficient adversarial images have been identified for the current iteration. If sufficient adversarial images have not been identified, logic flow 500 may return to block 514, "Generate Test Images." However, if sufficient adversarial images have been identified, logic flow 500 may proceed to block 524, "Input Adversarial Images." At block 524, the adversarial images may be provided as input to block 506, "Determine Current Training Set." Therefore, when logic flow 500 returns to block 506 from block 526, the current training set may be updated with the adversarial images. In various embodiments, this may enable the next iteration of DNN strengthening to begin.

[0050] Figure 6 One embodiment of a logic flow 600 is shown, which may represent operations that may be performed in various embodiments, such as in conjunction with an embodiment for generating an enhanced DNN 150. The logic flow 600 may represent the operations performed by Figure 1-4 Some or all of the operations performed by one or more components of the operating environments 100, 200, 300, 400, such as the DNN training system 102, the training set manager 104, the DNN training 106, or the adversarial image generator 108. The embodiments are not limited in this context.

[0051] exist Figure 6 In the illustrative embodiment shown in FIG, logic flow 600 may begin at block 602. At block 602, "Generate an adversarial image misclassified by a first deep neural network (DNN), the first DNN trained on a first training set of one or more sample images," an adversarial image misclassified by the first DNN trained on the first training set of one or more sample images is generated. For example, adversarial image 252-1 may be generated by adversarial image generator 108 based on trained DNN 304-1 inappropriately classifying adversarial image 252-1. In some instances, adversarial image 252-1 may be generated by altering one or more portions of a base image, such as sample image 212-1.

[0052] Proceeding to block 604, “Determine a second training set comprising adversarial images misclassified by the first DNN and one or more portions of the first training set of one or more sample images,” a second training set is determined, comprising adversarial images misclassified by the first DNN and one or more portions of the first training set of one or more sample images. For example, the training set manager 104 may determine the training set 216-2 to include the training set 216-1 and the adversarial image set 254-1.

[0053] At block 606, “Provide Second Training Set to Train Second DNN,” a second training set may be provided to train the second DNN. For example, training set 216-2 may be input to untrained DNN 302 to produce trained DNN 304-2. In various embodiments, trained DNN 304-2 may be output by DNN training system 102 as an enhanced DNN 150. In various such embodiments, this may be based on iteration tracker 306 determining that iteration parameter 201 has been satisfied.

[0054] Figure 7 An embodiment of a storage medium 700 is shown. The storage medium 700 may include any non-transitory computer-readable storage medium or machine-readable storage medium, such as an optical storage medium, a magnetic storage medium, or a semiconductor storage medium. In various embodiments, the storage medium 700 may include an article of manufacture. In some embodiments, the storage medium 700 may store computer-executable instructions, such as instructions for implementing the logic flows or operations described herein (see Figure 4 and Figure 5 Examples of computer-readable storage media or machine-readable storage media may include any tangible medium capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and the like. Examples of computer-executable instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, and the like. The embodiments are not limited in this context.

[0055] Figure 8 An embodiment of an exemplary computing architecture 800 that may be suitable for implementing various embodiments as previously described is shown. In various embodiments, the computing architecture 800 may include an electronic device or may be implemented as part of an electronic device. In some embodiments, the computing architecture 800 may represent, for example, an electronic device that implements Figure 1-4The computing architecture 800 may be a computer system that includes one or more components of the operating environments 100, 200, 300, and 400. In some embodiments, the computing architecture 800 may represent, for example, one or more portions or components of the DNN training system 102 that implement one or more embodiments described herein. The embodiments are not limited in this context.

[0056] As used herein, the terms "system," "component," and "module" are intended to refer to a computer-related entity, whether hardware, a combination of hardware and software, software, or software in execution, examples of which are provided by exemplary computing architecture 800. For example, a component can be, but is not limited to, the following: a process running on a processor, a processor, a hard drive, multiple storage drives (of optical and / or magnetic storage media), an object, executable code, a thread of execution, a program, and / or a computer. By way of illustration, both an application running on a server and the server can be components. One or more components can reside within a process and / or thread of execution, and components can be localized on a single computer and / or distributed across two or more computers. Furthermore, components can be communicatively coupled to each other via various types of communication media to coordinate operations. This coordination can involve unidirectional or bidirectional information exchange. For example, components can communicate information in the form of signals transmitted over a communication medium. This information can be implemented as signals assigned to various signal lines. In such an assignment, each message is a signal. However, further embodiments may alternatively employ data messages. Such data messages can be sent across various connections. Exemplary connections include parallel interfaces, serial interfaces, and bus interfaces.

[0057] The computing architecture 800 includes various common computing elements, such as one or more processors, multi-core processors, coprocessors, memory units, chipsets, controllers, peripherals, interfaces, oscillators, timing devices, video cards, audio cards, multimedia input / output (I / O) components, power supplies, etc. However, embodiments are not limited to implementation by the computing architecture 800.

[0058] like Figure 8 As shown in FIG, computing architecture 800 includes a processing unit 804, a system memory 806, and a system bus 808. Processing unit 804 can be any of various commercially available processors, including but not limited to: Athlon Poison Dragon He Haolong processor; applications, embedded or security processors; of Dragon Ball and Processor; IBM and Cell processor; Celeron Core(2) Itanium Pentium Xeon and Dual microprocessors, multi-core processors, and other multi-processor architectures can also be used as the processing unit 804.

[0059] The system bus 808 provides an interface for system components, including, but not limited to, the system memory 80 to the processing unit 804. The system bus 808 can be any of several types of bus structures that can interconnect to a memory bus (using or supporting a memory controller), a peripheral bus, and a local bus using one of various commercially available bus architectures. Interface adapters can be connected to the system bus 808 via a slot architecture. Example slot architectures can include, but are not limited to, Accelerated Graphics Port (AGP), Card Bus, (Extended) Industry Standard Architecture ((E)ISA), Micro Channel Architecture (MCA), Network User Bus (NuBus), Peripheral Component Interconnect (Extended) (PCI(X)), PCI Express, Personal Computer Memory Card International Association (PCMCIA) buses, and the like.

[0060] The system memory 806 may include various types of computer-readable storage media in the form of one or more higher-speed memory units, such as read-only memory (ROM), random-access memory (RAM), dynamic RAM (DRAM), double data rate DRAM (DDRAM), synchronous DRAM (SDRAM), static RAM (SRAM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory (e.g., one or more flash memory arrays), polymer memory such as ferroelectric polymer memory, Austenite memory, phase change or ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS) memory, magnetic or optical cards, device arrays such as redundant array of independent disks (RAID) drives, solid-state memory devices (e.g., USB memory, solid-state drive (SSD)), and any other type of storage medium suitable for storing information. Figure 8 In the illustrated embodiment shown in FIG, system memory 806 may include non-volatile memory 810 and / or volatile memory 812. A basic input / output system (BIOS) may be stored in non-volatile memory 810.

[0061] The computer 802 may include various types of computer-readable storage media in the form of one or more relatively high-speed memory units, including an internal (or external) hard disk drive (HDD) 814, a magnetic floppy disk drive (FDD) 818 for reading from or writing to a removable magnetic disk 816, and an optical drive 822 for reading from or writing to a removable optical disk 820 (e.g., a CD-ROM or DVD). The HDD 814, FDD 816, and optical drive 820 may be connected to the system bus 808 by a HDD interface 824, an FDD interface 826, and an optical drive interface 828, respectively. The HDD interface 824 for external drive implementations may include at least one or both of Universal Serial Bus (USB) and IEEE 994 interface technologies.

[0062] The drives and associated computer-readable media provide volatile and / or non-volatile storage of data, data structures, computer-executable instructions, etc. For example, a number of program modules may be stored in the drives and memory units 810, 812, including an operating system 830, one or more application programs 832, other program modules 834, and program data 836. In one embodiment, the one or more application programs 832, other program modules 834, and program data 836 may include, for example, various applications and / or components of the DNN training system 102, such as one or more portions of the training set manager 104, the DNN trainer 106, and / or the adversarial image generator 108.

[0063] A user can enter commands and information into the computer 802 through one or more wired / wireless input devices, such as a keyboard 838 and a pointing device such as a mouse 840. Other input devices might include a microphone, an infrared (IR) remote control, a radio frequency (RF) remote control, a game pad, a stylus, a card reader, a dongle, a fingerprint reader, a glove, a drawing tablet, a joystick, a keyboard, a retina reader, a touch screen (e.g., capacitive, resistive, etc.), a trackball, a trackpad, a sensor, a stylus, and the like. These and other input devices are typically connected to the processing unit 808 through an input device interface 842 coupled to the system bus 804, but may also be connected through other interfaces such as a parallel port, an IEEE 994 serial port, a game port, a USB port, an IR port, and the like.

[0064] A monitor 844 or other type of display device is also connected to the system bus 808 via an interface, such as a video adapter 846. The monitor 844 may be internal or external to the computer 802. In addition to the monitor 844, computers typically include other peripheral output devices such as speakers, printers, and the like.

[0065] Computer 802 can operate in a networked environment using logical connections to one or more remote computers (such as remote computer 848) via wired and / or wireless communications. In various embodiments, one or more migrations can occur via a networked environment. Remote computer 848 can be a workstation, server computer, router, personal computer, portable computer, microprocessor-based entertainment device, peer device, or other common network node, and typically includes many or all of the elements described with respect to computer 802, but only memory / storage device 850 is shown for simplicity. The depicted logical connections include wired / wireless connectivity to a local area network (LAN) 852 and / or a larger network such as a wide area network (WAN) 854. Such LAN and WAN networking environments are common in offices and companies and facilitate enterprise-wide computer networks such as intranets, all of which can be connected to global communication networks such as the Internet.

[0066] When used in a LAN networking environment, the computer 802 is connected to the LAN 852 through a wired and / or wireless communication network interface or adapter 856. The adapter 856 can facilitate wired and / or wireless communication to the LAN 852 and may also include a wireless access point provided thereon for communicating using the wireless functionality of the adapter 856.

[0067] When used in a WAN networking environment, the computer 802 may include a modem 1358, or be connected to a communication server on the WAN 1354, or have other means for establishing communications on the WAN 854, such as through the Internet. The modem 858 is connected to the system bus 808 via the input device interface 842 and may be an internal or external and / or wired and / or wireless device. In a networked environment, program modules depicted relative to the computer 802, or portions thereof, may be stored in the remote memory / storage device 850. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

[0068] The computer 802 is operable to communicate using the IEEE 802 family of standards with wired and wireless devices or entities, such as wireless devices operable to implement wireless communications (e.g., IEEE 802.16 over-the-air modulation techniques). This includes at least Wi-Fi (or Wireless Fidelity), WiMax, and Bluetooth. TMWireless technology, etc. Thus, communication can be a predefined structure like a conventional network, or simply ad hoc communication between at least two devices. Wi-Fi networks use radio technologies called IEEE 802.11x (a, b, n, g, etc.) to provide secure, reliable, and fast wireless connectivity. Wi-Fi networks can be used to connect computers to each other, to the Internet, and to wired networks (which use IEEE 802.3 related media and functions).

[0069] Figure 9 A block diagram of an exemplary communication architecture 900 suitable for implementing various embodiments as previously described (such as virtual machine migration) is shown. The communication architecture 900 includes various common communication elements, such as transmitters, receivers, transceivers, radios, network interfaces, baseband processors, antennas, amplifiers, filters, power supplies, etc. However, the embodiments are not limited to implementation using the communication architecture 900.

[0070] like Figure 9 As shown in FIG, the communication architecture 900 includes one or more clients 902 and servers 904. The clients 902 and servers 904 are operatively connected to one or more respective client data stores 908 and server data stores 910 that can be used to store information local to the respective clients 902 and servers 904 (such as cookies and / or associated contextual information). In various embodiments, any of the servers 904 can implement the logic flows or operations described herein in conjunction with storing data received from any of the clients 902 on any of the server data stores 910. Figure 7 One or more of the storage media 700.

[0071] The client 902 and the server 904 can communicate information between each other using a communication framework 906. The communication framework 906 can implement any well-known communication technologies and protocols. The communication framework 906 can be implemented as a packet-switched network (e.g., a public network such as the Internet, a private network such as a corporate intranet, etc.), a circuit-switched network (e.g., a public switched telephone network), or a combination of packet-switched and circuit-switched networks (using appropriate gateways and switches).

[0072] The communication framework 906 can implement various network interfaces arranged to receive, communicate and connect to the communication network. The network interface can be considered as a dedicated form of input and output interface. The network interface can adopt a connection protocol, including but not limited to direct connection, Ethernet (e.g., thick, thin, twisted pair 10 / 100 / 1900Base T, etc.), token ring, wireless network interface, cellular network interface, IEEE 802.11ax network interface, IEEE 802.16 network interface, IEEE 802.20 network interface, etc. In addition, multiple network interfaces can be used to engage with various communication network types. For example, multiple network interfaces can be used to allow communication on broadcast, multicast and unicast networks. If the processing requirements specify greater speed and capacity, a distributed network controller architecture can be similarly adopted to share load balancing and otherwise increase the communication bandwidth required by the client 902 and server 904. The communication network can be any of wired and / or wireless networks and a combination of wired and wireless networks, including but not limited to direct interconnections, secure custom connections, private networks (e.g., corporate intranets), public networks (e.g., the Internet), personal area networks (PANs), local area networks (LANs), metropolitan area networks (MANs), operating missions as nodes in an internetwork (OMNI), wide area networks (WANs), wireless networks, cellular networks, and other communication networks.

[0073] Each embodiment can be realized using hardware elements, software elements or the combination of the two.The example of hardware elements may include processors, microprocessors, circuits, circuit elements (for example, transistors, resistors, capacitors, inductors, etc.), integrated circuits, application specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), logic gates, registers, semiconductor devices, chips, microchips, chipsets, etc. The example of software may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, processes, software interfaces, application program interfaces (APIs), instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols or any combination thereof. Determine whether to use hardware elements and / or software elements to realize that an embodiment can vary according to any number of factors, such as desired computing rate, power level, thermal tolerance, processing cycle budget, input data rate, output data rate, memory resources, data bus speed and other design or performance constraints.

[0074] One or more aspects of at least one embodiment may be implemented by representative instructions representing various logic in a processor stored on a machine-readable medium, which, when read by a machine, causes the machine to manufacture logic for performing the techniques described herein. Such representations, known as "IP cores," may be stored on tangible machine-readable media and supplied to various customers or production facilities to be loaded into manufacturing machines that actually manufacture the logic or processor. For example, some embodiments may be implemented using machine-readable media or articles of manufacture that may store instructions or a set of instructions that, when executed by a machine, may cause the machine to perform methods and / or operations according to the embodiments. Such machines may include, for example, any suitable processing platform, computing platform, computing device, processing device, computing system, processing system, computer, processor, etc., and may be implemented using any suitable combination of hardware and / or software. The machine-readable medium or article of manufacture may include, for example, any suitable type of memory unit, memory device, memory article, memory medium, storage device, storage article, storage medium, and / or storage unit, such as memory, removable or non-removable media, erasable or non-erasable media, writable or rewritable media, digital or analog media, hard disk, floppy disk, compact disk read only memory (CD-ROM), compact disk recordable (CD-R), compact disk rewritable (CD-W), optical disk, magnetic media, magneto-optical media, removable memory cards or disks, various types of digital versatile disks (DVDs), magnetic tape, tape cassettes, etc. The instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, etc., implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0075] The following examples relate to further embodiments, from which numerous permutations and configurations will become apparent.

[0076] Example 1 is a device for performing an analysis attack, the device comprising: a memory; and logic, at least a portion of which is implemented in a circuit coupled to the memory, the logic configured to: generate an adversarial image misclassified by a first deep neural network (DNN), the first DNN trained using a first training set of one or more sample images; determine a second training set comprising the adversarial image misclassified by the first DNN and one or more portions of the first training set of one or more sample images; and provide the second training set to train a second DNN.

[0077] Example 2 includes the subject matter of Example 1, wherein generating an adversarial image misclassified by the first DNN comprises logic for: changing one or more portions of a base image to generate a test image; providing the test image to the first DNN for classification; and identifying the test image as an adversarial image misclassified by the first DNN when the classification of the test image is different from the target classification.

[0078] Example 3 includes the subject matter of Example 2, wherein generating the adversarial image misclassified by the first DNN comprises logic for identifying the test image as an adversarial image misclassified by the first DNN when the classification of the test image is different from the target classification and the classification of the test image is associated with a confidence level that satisfies a threshold confidence level.

[0079] Example 4 includes the subject matter of Example 2, wherein generating the adversarial image misclassified by the first DNN comprises logic for: changing one or more portions of the test image when the classification of the test image is the same as the target classification.

[0080] Example 5 includes the subject matter of Example 2, the logic being to alter one or more portions of bytecode in an assembly file used to generate the base image based on the one or more altered portions of the base image when the classification of the test image differs from the target classification.

[0081] Example 6 includes the subject matter of Example 5, the logic to: identify one or more portions of the bytecode to change based on a sliding window technique.

[0082] Example 7 includes the subject matter of Example 6, wherein a sliding window technique utilizes the base image and the one or more changed portions of the base image.

[0083] Example 8 includes the subject matter of Example 2, the base images comprising sample images in the first training set, and the target classification comprising a classification of the sample images.

[0084] Example 9 includes the subject matter of Example 8, wherein the base image and the test image are close according to a distance metric.

[0085] Example 10 includes the subject matter of Example 2, the target classification comprising a classification associated with white noise, a classification different from any classification associated with the first training set, or an unclassifiable classification.

[0086] Example 11 includes the subject matter of Example 2, generating an adversarial image misclassified by the first DNN comprising logic for utilizing a fast gradient sign method to alter one or more portions of a base image to generate a test image.

[0087] Example 12 includes the subject matter of Example 2, generating an adversarial image misclassified by the first DNN comprising logic for utilizing an evolutionary algorithm to alter one or more portions of a base image to generate the test image.

[0088] Example 13 includes the subject matter of Example 12, wherein the evolutionary algorithm is used to exploit a multidimensional archive of phenotypic elites.

[0089] Example 14 includes the subject matter of Example 2, wherein generating an adversarial image that is misclassified by the first DNN comprises logic for minimizing a Chebyshev or L-infinity distance between a base image and a test image when one or more portions of the base image are altered to generate the test image.

[0090] Example 15 includes the subject matter of Example 1, wherein the first and second DNNs are trained to detect malware.

[0091] Example 16 includes the subject matter of Example 1, generating at least one sample image in the first training set from the bytecode.

[0092] Example 17 includes the subject matter of Example 16, the logic being to transform the bytecodes into pixel intensities to generate at least one sample image in the first training set.

[0093] Example 18 includes the subject matter of Example 16, the bytecode comprising malware bytecode.

[0094] Example 19 includes the subject matter of Example 1, the logic being to: generate a second adversarial image misclassified by the second DNN; determine a third training set comprising the second adversarial image misclassified by the second DNN and one or more portions of the second training set; and provide the third training set to train a third DNN.

[0095] Example 20 includes the subject matter of Example 19, wherein one or more portions of the second training set include adversarial images misclassified by the first DNN.

[0096] Example 21 includes the subject matter of Example 19, the one or more portions of the second training set comprising at least one sample image of the one or more sample images in the first training set.

[0097] Example 22 includes the subject matter of Example 1, wherein the first and second DNNs utilize a multilayer perceptron architecture.

[0098] Example 23 includes the subject matter of Example 22, wherein the logic is for changing multiple layers in a multilayer perceptron architecture.

[0099] Example 24 includes the subject matter of Example 1, wherein the first and second DNNs utilize convolutional neural network architectures.

[0100] Example 25 includes the subject matter of Example 24, with logic for changing the filter size in a convolutional neural network.

[0101] Example 26 is at least one non-transitory computer-readable medium comprising a set of instructions that, in response to the instructions being executed on a computing device, cause the computing device to: generate an adversarial image misclassified by a first deep neural network (DNN), the first DNN trained using a first training set of one or more sample images; determine a second training set comprising the adversarial image misclassified by the first DNN and one or more portions of the first training set of one or more sample images; and provide the second training set to train a second DNN.

[0102] Example 27 includes the subject matter of Example 26, wherein generating an adversarial image misclassified by the first DNN comprises instructions that, in response to being executed on the computing device, cause the computing device to: change one or more portions of a base image to generate a test image; provide the test image to the first DNN for classification; and when the classification of the test image is different from the target classification, identify the test image as an adversarial image misclassified by the first DNN.

[0103] Example 28 includes the subject matter of Example 27, wherein generating an adversarial image misclassified by the first DNN comprises instructions that, in response to being executed on the computing device, cause the computing device to: identify the test image as an adversarial image misclassified by the first DNN when the classification of the test image is different from the target classification and the classification of the test image is associated with a confidence level that satisfies a threshold confidence level.

[0104] Example 29 includes the subject matter of Example 27, wherein generating an adversarial image misclassified by the first DNN comprises instructions that, in response to being executed on the computing device, cause the computing device to: change one or more portions of the test image when the classification of the test image is the same as the target classification.

[0105] Example 28 includes the subject matter of Example 27, wherein generating an adversarial image misclassified by the first DNN comprises instructions that, in response to being executed on the computing device, cause the computing device to: when a classification of the test image is different from a target classification, change one or more portions of bytecode in an assembly file used to generate the base image based on the one or more changed portions of the base image.

[0106] Example 31 includes the subject matter of Example 30, comprising instructions that, in response to being executed on the computing device, cause the computing device to: identify one or more portions of bytecode to change based on a sliding window technique.

[0107] Example 32 includes the subject matter of Example 31, wherein a sliding window technique utilizes the base image and the one or more changed portions of the base image.

[0108] Example 33 includes the subject matter of Example 27, the base images comprising sample images in the first training set, and the target classification comprising a classification of the sample images.

[0109] Example 34 includes the subject matter of Example 33, wherein the base image and the test image are close according to a distance metric.

[0110] Example 35 includes the subject matter of Example 27, the target classification comprising a classification associated with white noise, a classification different from any classification associated with the first training set, or an unclassifiable classification.

[0111] Example 36 includes the subject matter of Example 27, wherein generating an adversarial image misclassified by the first DNN comprises instructions that, in response to being executed on the computing device, cause the computing device to: utilize a fast gradient sign method to change one or more portions of a base image to generate a test image.

[0112] Example 37 includes the subject matter of Example 27, wherein generating an adversarial image misclassified by the first DNN comprises instructions that, in response to being executed on the computing device, cause the computing device to: utilize an evolutionary algorithm to alter one or more portions of a base image to generate a test image.

[0113] Example 38 includes the subject matter of Example 37, wherein the evolutionary algorithm is used to exploit a multidimensional archive of phenotypic elites.

[0114] Example 39 includes the subject matter of Example 27, wherein generating an adversarial image misclassified by the first DNN comprises instructions that, in response to being executed on the computing device, cause the computing device to: minimize a Chebyshev or L-infinity distance between a base image and a test image when one or more portions of the base image are altered to generate the test image.

[0115] Example 40 includes the subject matter of Example 26, wherein the first and second DNNs are trained to detect malware.

[0116] Example 41 includes the subject matter of Example 26, generating at least one sample image in the first training set from the bytecode.

[0117] Example 42 includes the subject matter of Example 41, wherein generating an adversarial image misclassified by the first DNN comprises instructions that, in response to being executed on the computing device, cause the computing device to: transform bytecodes into pixel intensities to generate at least one sample image in the first training set.

[0118] Example 43 includes the subject matter of Example 41, the bytecode comprising malware bytecode.

[0119] Example 44 includes the subject matter of Example 26, comprising instructions that, in response to being executed on the computing device, cause the computing device to: generate a second adversarial image misclassified by the second DNN; determine a third training set comprising the second adversarial image misclassified by the second DNN and one or more portions of the second training set; and provide the third training set to train a third DNN.

[0120] Example 45 includes the subject matter of Example 44, wherein one or more portions of the second training set include adversarial images misclassified by the first DNN.

[0121] Example 46 includes the subject matter of Example 44, the one or more portions of the second training set comprising at least one of the one or more sample images in the first training set.

[0122] Example 47 includes the subject matter of Example 26, wherein the first and second DNNs utilize a multilayer perceptron architecture.

[0123] Example 48 includes the subject matter of Example 47, comprising instructions that, in response to being executed on the computing device, cause the computing device to: change a plurality of layers in a multilayer perceptron architecture.

[0124] Example 49 includes the subject matter of Example 26, wherein the first and second DNNs utilize a convolutional neural network architecture.

[0125] Example 50 includes the subject matter of Example 49, comprising instructions that, in response to being executed on the computing device, cause the computing device to: change a filter size in a convolutional neural network.

[0126] Example 51 is a computer-implemented method comprising: generating an adversarial image misclassified by a first deep neural network (DNN), the first DNN being trained using a first training set of one or more sample images; determining a second training set comprising the adversarial image misclassified by the first DNN and one or more portions of the first training set of one or more sample images; and providing the second training set to train a second DNN.

[0127] Example 52 includes the subject matter of Example 51, wherein generating an adversarial image misclassified by the first DNN comprises: changing one or more portions of a base image to generate a test image; providing the test image to the first DNN for classification; and when the classification of the test image is different from the target classification, identifying the test image as an adversarial image misclassified by the first DNN.

[0128] Example 53 includes the subject matter of Example 52, wherein generating the adversarial image misclassified by the first DNN comprises: when the classification of the test image is different from the target classification and the classification of the test image is associated with a confidence level that satisfies a threshold confidence level, identifying the test image as an adversarial image misclassified by the first DNN.

[0129] Example 54 includes the subject matter of Example 52, wherein generating the adversarial image misclassified by the first DNN comprises: when the classification of the test image is the same as the target classification, changing one or more portions of the test image.

[0130] Example 55 includes the subject matter of Example 52, including changing one or more portions of bytecode in an assembly file used to generate the base image based on the one or more changed portions of the base image when the classification of the test image is different from the target classification.

[0131] Example 56 includes the subject matter of Example 55, comprising identifying one or more portions of the bytecode to change based on a sliding window technique.

[0132] Example 57 includes the subject matter of Example 56, wherein a sliding window technique utilizes one or more changing portions of a base image and the base image.

[0133] Example 58 includes the subject matter of Example 52, the base images comprising sample images in the first training set, and the target classification comprising a classification of the sample images.

[0134] Example 59 includes the subject matter of Example 58, wherein the base image and the test image are close according to a distance metric.

[0135] Example 60 includes the subject matter of Example 52, the target classification comprising a classification associated with white noise, a classification different from any classification associated with the first training set, or an unclassifiable classification.

[0136] Example 61 includes the subject matter of Example 52, wherein generating the adversarial image misclassified by the first DNN comprises: utilizing a fast gradient sign method to alter one or more portions of a base image to generate the test image.

[0137] Example 62 includes the subject matter of Example 52, generating the adversarial image misclassified by the first DNN comprising: utilizing an evolutionary algorithm to alter one or more portions of the base image to generate the test image.

[0138] Example 63 includes the subject matter of Example 62, wherein the evolutionary algorithm is used to exploit a multidimensional archive of phenotypic elites.

[0139] Example 64 includes the subject matter of Example 52, generating an adversarial image misclassified by the first DNN comprising: minimizing a Chebyshev or L-infinity distance between the base image and the test image when one or more portions of the base image are altered to generate the test image.

[0140] Example 65 includes the subject matter of Example 51, wherein the first and second DNNs are trained to detect malware.

[0141] Example 66 includes the subject matter of Example 51, generating at least one sample image in the first training set from the bytecode.

[0142] Example 67 includes the subject matter of Example 66, comprising transforming the bytecodes into pixel intensities to generate at least one sample image in the first training set.

[0143] Example 68 includes the subject matter of Example 66, the bytecode comprising malware bytecode.

[0144] Example 69 includes the subject matter of Example 51, including: generating a second adversarial image misclassified by the second DNN; determining a third training set comprising the second adversarial image misclassified by the second DNN and one or more portions of the second training set; and providing the third training set to train a third DNN.

[0145] Example 70 includes the subject matter of Example 69, wherein one or more portions of the second training set include adversarial images misclassified by the first DNN.

[0146] Example 71 includes the subject matter of Example 69, the one or more portions of the second training set comprising at least one of the one or more sample images in the first training set.

[0147] Example 72 includes the subject matter of Example 51, wherein the first and second DNNs utilize a multilayer perceptron architecture.

[0148] Example 73 includes the subject matter of Example 72, including altering a plurality of layers in a multilayer perceptron architecture.

[0149] Example 74 includes the subject matter of Example 51, wherein the first and second DNNs utilize a convolutional neural network architecture.

[0150] Example 75 includes the subject matter of Example 74, including changing the filter size in a convolutional neural network.

[0151] Example 75 includes the subject matter of Example 74, including changing the filter size in a convolutional neural network.

[0152] Example 76 is an apparatus for performing an analysis attack, the apparatus comprising: a device for generating an adversarial image misclassified by a first deep neural network (DNN), the first DNN being trained using a first training set of one or more sample images; a device for determining a second training set comprising the adversarial image misclassified by the first DNN and one or more portions of the first training set of one or more sample images; and a device for providing the second training set to train a second DNN.

[0153] Example 77 includes the subject matter of Example 76, wherein generating an adversarial image misclassified by the first DNN comprises: means for changing one or more portions of a base image to generate a test image; means for providing the test image to the first DNN for classification; and means for identifying the test image as an adversarial image misclassified by the first DNN when the classification of the test image is different from the target classification.

[0154] Example 78 includes the subject matter of Example 77, wherein generating the adversarial image misclassified by the first DNN comprises: means for identifying the test image as an adversarial image misclassified by the first DNN when the classification of the test image is different from the target classification and the classification of the test image is associated with a confidence level that satisfies a threshold confidence level.

[0155] Example 79 includes the subject matter of Example 77, wherein generating the adversarial image misclassified by the first DNN comprises means for changing one or more portions of the test image when the classification of the test image is the same as the target classification.

[0156] Example 80 includes the subject matter of Example 77, including means for changing one or more portions of bytecode in an assembly file used to generate a base image based on one or more changed portions of the base image when a classification of a test image differs from a target classification.

[0157] Example 81 includes the subject matter of Example 80, comprising means for identifying one or more portions of bytecode to change based on a sliding window technique.

[0158] Example 82 includes the subject matter of Example 81, wherein a sliding window technique utilizes one or more changing portions of a base image and the base image.

[0159] Example 83 includes the subject matter of Example 77, the base images comprising sample images in the first training set, and the target classification comprising a classification of the sample images.

[0160] Example 84 includes the subject matter of Example 83, wherein the base image and the test image are close according to a distance metric.

[0161] Example 85 includes the subject matter of Example 77, the target classification comprising a classification associated with white noise, a classification different from any classification associated with the first training set, or an unclassifiable classification.

[0162] Example 86 includes the subject matter of Example 77, wherein generating an adversarial image misclassified by the first DNN comprises means for utilizing a fast gradient sign method to alter one or more portions of a base image to generate a test image.

[0163] Example 87 includes the subject matter of Example 77, wherein generating an adversarial image misclassified by the first DNN comprises means for utilizing an evolutionary algorithm to alter one or more portions of a base image to generate a test image.

[0164] Example 88 includes the subject matter of Example 87, wherein the evolutionary algorithm is used to exploit a multidimensional archive of phenotypic elites.

[0165] Example 89 includes the subject matter of Example 77, wherein generating an adversarial image that is misclassified by the first DNN comprises means for minimizing a Chebyshev or L-infinity distance between the base image and the test image when one or more portions of the base image are altered to generate the test image.

[0166] Example 90 includes the subject matter of Example 76, wherein the first and second DNNs are trained to detect malware.

[0167] Example 91 includes the subject matter of Example 76, generating at least one sample image in the first training set from the bytecode.

[0168] Example 92 includes the subject matter of Example 91, comprising means for transforming bytecodes into pixel intensities to generate at least one sample image in the first training set.

[0169] Example 93 includes the subject matter of Example 91, the bytecode comprising malware bytecode.

[0170] Example 94 includes the subject matter of Example 76, including: means for generating a second adversarial image misclassified by a second DNN; means for determining a third training set comprising the second adversarial image misclassified by the second DNN and one or more portions of the second training set; and means for providing the third training set to train a third DNN.

[0171] Example 95 includes the subject matter of Example 94, wherein one or more portions of the second training set include adversarial images misclassified by the first DNN.

[0172] Example 96 includes the subject matter of Example 94, the one or more portions of the second training set comprising at least one of the one or more sample images in the first training set.

[0173] Example 97 includes the subject matter of Example 76, wherein the first and second DNNs utilize a multilayer perceptron architecture.

[0174] Example 98 includes the subject matter of Example 97, including means for varying a plurality of layers in a multilayer perceptron architecture.

[0175] Example 99 includes the subject matter of Example 76, wherein the first and second DNNs utilize a convolutional neural network architecture.

[0176] Example 100 includes the subject matter of Example 99, including means for changing filter size in a convolutional neural network.

[0177] The foregoing description of the exemplary embodiments has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and variations are possible in light of the foregoing disclosure. The scope of the disclosure is limited not by this detailed description but by the appended claims. Future applications claiming priority from the present application may claim the disclosed subject matter in different ways and may generally include any combination of one or more limitations as variously disclosed or otherwise illustrated herein.

Claims

1. A computer-implemented method comprising: Generating an adversarial image misclassified by a first deep neural network, wherein the first deep neural network is trained using a first training set of one or more sample images, wherein generating the adversarial image misclassified by the first deep neural network includes: changing one or more portions of a base image to generate a test image; When the classification of the test image is different from the target classification, identifying the test image as the adversarial image misclassified by the first deep neural network; and When the classification of the test image is the same as the target classification, changing one or more portions of the test image; determining a second training set comprising one or more portions of the first training set of the adversarial images and the one or more sample images that were misclassified by the first deep neural network; and The second training set is provided to train a second deep neural network.

2. The computer-implemented method of claim 1 , generating the adversarial image misclassified by the first deep neural network comprising: The test image is provided to the first deep neural network for classification.

3. The computer-implemented method of claim 2, generating the adversarial image misclassified by the first deep neural network comprising: When the classification of the test image is different from the target classification and the classification of the test image is associated with a confidence level that satisfies a threshold confidence level, the test image is identified as the adversarial image misclassified by the first deep neural network.

4. The computer-implemented method of claim 2, comprising: When the classification of the test image is different from the target classification, one or more portions of bytecodes in an assembly file used to generate the base image are changed based on the one or more changed portions of the base image.

5. The computer-implemented method of claim 4 , comprising: One or more portions of the bytecode to be changed are identified based on a sliding window technique.

6. The computer-implemented method of claim 5, the sliding window technique being used to utilize the base image and one or more altered portions of the base image.

7. The computer-implemented method of claim 2, the base images comprising sample images in the first training set, and the target classification comprising a classification of the sample images.

8. The computer-implemented method of claim 7, wherein: The base image and the test image are close according to a distance metric.

9. The computer-implemented method of claim 2, the target classification comprising a classification associated with white noise, a classification different from any classification associated with the first training set, or an unclassifiable classification.

10. The computer-implemented method of claim 2, generating the adversarial image misclassified by the first deep neural network comprises: One or more portions of the base image are altered using a fast gradient sign method to generate the test image.

11. The computer-implemented method of claim 2, generating the adversarial image misclassified by the first deep neural network comprises: An evolutionary algorithm is used to alter one or more portions of the base image to generate the test image.

12. The computer-implemented method of claim 11, wherein the evolutionary algorithm is used to utilize a multidimensional archive of phenotypic elites.

13. The computer-implemented method of claim 2, generating the adversarial image misclassified by the first deep neural network comprises: When one or more portions of the base image are altered to generate the test image, a Chebyshev or L-infinity distance between the base image and the test image is minimized.

14. The computer-implemented method of claim 1, the first deep neural network and the second deep neural network are trained to detect malware.

15. The computer-implemented method of claim 1, wherein at least one sample image in the first training set is generated from bytecode.

16. The computer-implemented method of claim 15, comprising: The bytecodes are transformed into pixel intensities to generate the at least one sample image in the first training set.

17. The computer-implemented method of claim 15, the bytecode comprising malware bytecode.

18. The computer-implemented method of claim 1 , comprising: generating a second adversarial image that is misclassified by the second deep neural network; determining a third training set comprising the second adversarial images misclassified by the second deep neural network and one or more portions of the second training set; and The third training set is provided to train a third deep neural network.

19. The computer-implemented method of claim 18, wherein one or more portions of the second training set include adversarial images that were misclassified by the first deep neural network.

20. The computer-implemented method of claim 18, the one or more portions of the second training set comprising at least one sample image of the one or more sample images in the first training set.

21. The computer-implemented method of claim 1, wherein the first deep neural network and the second deep neural network are configured to utilize a multi-layer perceptron architecture.

22. The computer-implemented method of claim 1, wherein the first deep neural network and the second deep neural network are configured to utilize a convolutional neural network architecture.

23. Computer device comprising means for performing the method of any one of claims 1 to 22.

24. A machine-readable medium comprising code which, when executed, causes a machine to perform the method of any one of claims 1 to 22.

Images