A malware identification method and system based on an innocent until proven guilty IUPG model

Abstract

Techniques are disclosed for providing an innocent until proven guilty (IUPG) solution for building and using deep learning models that are resistant to adversaries and resistant to false positives. In some embodiments, a system, process, and / or computer program product includes storing a collection comprising one or more innocent until proven guilty (IUPG) models for static analysis of samples; performing static analysis of content associated with a sample, wherein performing the static analysis includes using at least one stored IUPG model; and determining, based at least in part on the static analysis of the content associated with the sample, that the sample is malicious, and in response to determining that the sample is malicious, performing an action based on a security policy.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 034,843, filed June 4, 2020, entitled “INNOCENT UNTIL PROVEN GUILTY (IUPG): BUILDING ADVERSARY RESISTANT AND FALSE POSITIVE RESISTANT DEEP LEARNING MODELS,” which is incorporated by reference herein for all purposes. BACKGROUND

[0003] Malware is a general term that is often used to refer to software that is malicious (e.g., including a variety of hostile, intrusive, and / or otherwise unwanted software). Malware can be in the form of code, scripts, active content, and / or other software. Example uses of malware include disrupting computer and / or network operations, stealing proprietary information (e.g., confidential information such as identity, financial, and / or intellectual property related information), and / or gaining access to private / proprietary computer systems and / or computer networks. Unfortunately, as technology develops to help detect and mitigate malware, nefarious authors find ways to circumvent such efforts. Thus, there is a continuing need for improvements in techniques for identifying and mitigating malware. BRIEF DESCRIPTION OF DRAWINGS

[0004] Various embodiments of the application are disclosed in the following detailed description and the accompanying drawings.

[0005] Figure 1 An example of an environment in which a malicious application (“malware”) is detected and prevented from causing harm is illustrated.

[0006] Figure 2A An embodiment of a data appliance is illustrated.

[0007] Figure 2B A functional illustration of a logical component of a data appliance embodiment.

[0008] Figure 3 An example of a logical component that can be included in a system for analyzing a sample is illustrated.

[0009] Figure 4 A portion of an example embodiment of a threat engine is illustrated.

[0010] Figure 5A is a block diagram of an IUPG component enhanced on an abstract network

[0011] ​Figure 5B FIGURE illustrates an example of using a IUPG network, according to some embodiments.

[0012] Figure 5C FIGURE illustrates the expected force for on-target and off-target samples.

[0013] Figure 6 FIGURE illustrates the function for (A) MNIST and Fashion MNIST, (B) JS, and (C) URL, according to some embodiments .

[0014] Figure 7A FIGURE illustrates Table 1, which includes the malicious JS classification test set over all non-malignant classes FNR.

[0015] Figure 7B FIGURE illustrates Table 2, which includes the image classification noiseless test set error percentage.

[0016] Figure 7C FIGURE illustrates Table 3, which includes the image classification test set when the test set contains Gaussian noise images and the model was trained without noise error percentage.

[0017] Figure 7D FIGURE illustrates Table 4, which includes the malicious URL classification test set FNR.

[0018] Figure 7E FIGURE illustrates Table 5, which includes the detection with threshold configured by the 0.005% test set FPR organized by VTS.

[0019] Figure 8 FIGURE illustrates OOD attack simulation results.

[0020] Figure 9 FIGURE illustrates Table 6, which includes additional attack simulation results.

[0021] Figures 10A-10B FIGURE illustrates the accuracy over correctly classified test images versus the scaling factor of FGSM perturbations.

[0022] Figure 11 is an example t-SNE visualization of the U vector space, according to some embodiments.

[0023] Figures 12A-12C Various examples of additional attacks that can evade detection by existing malware detection solutions are provided.

[0024] Figure 13FIG. illustrates an IPUG framework for malware JavaScript classification, according to some embodiments.

[0025] Figure 14 is an example of a process for performing sample static analysis using an innocent until proven guilty (IUPG) model for malware classification, according to some embodiments.

[0026] Figure 15 is an example of a process for generating an innocent until proven guilty (IUPG) model for malware classification, according to some embodiments. DETAILED DESCRIPTION

[0027] The application can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product; and / or a processor, such as a processor configured to execute instructions stored on and / or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the application can take, can be referred to as techniques. In general, the order of the steps of disclosed processes can be altered, except where such alteration would necessarily affect the order of performing the steps. Unless otherwise indicated, components such as processors or memories described as being configured to perform a task can be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term 'processor' refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

[0028] A detailed description of one or more embodiments of the application is provided below along with accompanying figures that illustrate the principles of the application. The application is described in connection with such embodiments, but the application is not limited to any embodiment. The scope of the application is limited only by the claims and the application encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the application. These details are provided for the purpose of example and the application can be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the application has not been described in detail so that the application is not unnecessarily obscured.

[0029] Firewall technology overview

[0030] Firewalls generally protect networks from unauthorized access while permitting authorized communications to pass through the firewall. Firewalls are often devices, sets of devices, or software executing on devices that provide firewall functionality for network access. For example, a firewall can be integrated into an operating system of a device (e.g., a computer, a smart phone, or other type of device capable of network communication). Firewalls can also be integrated into or executed as one or more software applications on various types of devices, such as computer servers, gateways, network / routing devices (e.g., network routers), and data appliances (e.g., security appliances or other types of specialized devices), and in various implementations, certain operations can be implemented in specialized hardware, such as ASICs or FPGAs.

[0031] Firewalls generally deny or permit network transmissions based on a set of rules. These sets of rules are often referred to as policies (e.g., network policies or network security policies). For example, a firewall can filter inbound traffic by applying a set of rules or policies to prevent unwanted external traffic from reaching a protected device. A firewall can also filter outbound traffic (e.g., allow, block, monitor, notify, or log, and / or other actions can be specified in firewall rules or firewall policies that can be triggered based on various criteria, as described herein) by applying a set of rules or policies. A firewall can also filter local network (e.g., intranet) traffic by similarly applying a set of rules or policies.

[0032] Security devices (e.g., security appliances, security gateways, security services, and / or other security devices) can include various security functionality (e.g., firewall, anti-malware, intrusion prevention / detection, data loss prevention (DLP), and / or other security functionality), networking functionality (e.g., routing, quality of service (QoS), workload balancing of network-related resources, and / or other networking functionality), and / or other functionality. For example, routing functionality can be based on source information (e.g., IP address and port), destination information (e.g., IP address and port), and protocol information.

[0033] Basic packet filtering firewalls filter network communication traffic by inspecting individual packets transmitted on a network (e.g., packet filtering firewalls or first generation firewalls, which are stateless packet filtering firewalls). Stateless packet filtering firewalls generally inspect individual packets themselves and apply rules based on the inspected packets (e.g., using a combination of source and destination address information, protocol information, and port numbers of the packets).

[0034] Application firewalls can also perform application-layer filtering (e.g., application-layer filtering firewalls or second generation firewalls, which operate at the application layer of the TCP / IP stack). Application-layer filtering firewalls or application firewalls can generally identify certain applications and protocols (e.g., web browsing using the hypertext transfer protocol (HTTP), domain name system (DNS) requests, file transfers using the file transfer protocol (FTP), and various other types of applications and other protocols, such as remote login, DHCP, TCP, UDP, and TFTP (GSS)). For example, application firewalls can block unauthorized protocols that attempt to communicate over standard ports (e.g., unauthorized / out-of-policy protocols that attempt to sneak in by using a non-standard port for the protocol can generally be identified using an application firewall).

[0035] Stateful firewalls can also perform state-based packet inspection, in which each packet is inspected within the context of a series of packets associated with a flow of packets for a network transmission. This firewall technique is generally referred to as stateful packet inspection, as it maintains a record of all connections passing through the firewall and is able to determine whether a packet is the start of a new connection, part of an existing connection, or an invalid packet. For example, connection state itself can be one of the criteria that triggers rules within a policy.

[0036] As discussed above, advanced or next-generation firewalls can perform stateless and stateful packet filtering as well as application-layer filtering. Next-generation firewalls can also perform additional firewall techniques. For example, certain newer firewalls, sometimes referred to as advanced or next-generation firewalls, can also identify users and content (e.g., next-generation firewalls). In particular, certain next-generation firewalls are extending the list of applications that these firewalls can automatically identify to thousands of applications. Examples of such next-generation firewalls are commercially available from Palo Alto Networks, Inc. (e.g., Palo Alto Networks' PA Series firewalls). For example, Palo Alto Networks' next-generation firewalls enable enterprises to identify and control applications, users, and content using a variety of identification technologies, such as the following: APP-ID for accurate application identification, User-ID for user identification (e.g., by user or user group), and Content-ID for real-time content scanning (e.g., to control web surfing and limit data and file transfers). These identification technologies allow enterprises to safely enable application usage with business-relevant concepts, rather than following the traditional approach provided by traditional port-blocking firewalls. In addition, the specialized hardware used for next-generation firewalls (e.g., implemented as specialized appliances) generally provides higher levels of application inspection performance than software executing on general-purpose hardware (e.g., such as security appliances provided by Palo Alto Networks, Inc. that use specialized, function-specific processing that is tightly integrated with single-pass software engines to maximize network throughput while minimizing latency).

[0037] Advanced or next-generation firewalls can also be implemented using virtualized firewalls. Examples of such next-generation firewalls are commercially available from Palo Alto Networks, Inc. (e.g., Palo Alto Networks' VM Series firewalls, which support a variety of commercial virtualization environments, including, for example, VMware® ESXi™ and NSX™, Citrix® Netscaler SDX™, KVM / OpenStack (Centos / RHEL, Ubuntu®), and Amazon Web Services (AWS)). For example, virtualized firewalls can support similar or identical advanced threat prevention features available in next-generation firewalls and physical form factor appliances, allowing enterprises to safely enable application flow into and through their private, public, and hybrid cloud computing environments. Automation features such as VM monitoring, dynamic address groups, and REST-based APIs allow enterprises to proactively monitor VM changes that dynamically feed this context into security policies, eliminating potential policy lag when VM changes occur.

[0038] Example Environment

[0039] Figure 1 An example of an environment in which a malicious application ("malware") is detected and prevented from causing harm is illustrated. As will be described in greater detail below, malware classification (e.g., made by security platform 122) can be shared and / or refined differently between various entities included in the environment shown in FIG. 1. And, using the techniques described herein, devices such as endpoint client devices 104-110 can be protected from such malware. Figure 1

[0040] The term "application" is used throughout the specification to refer to a program, suite of programs, manifest, package, etc. regardless of form / platform. An "application" (also referred to herein as a "sample") can be a standalone file (e.g., a calculator application having a file name "calculator.apk" or "calculator.exe") and can also be a standalone component of another application (e.g., a mobile ad SDK or library embedded within a calculator application).

[0041] "Malware" as used herein refers to an application that engages in behavior that a user would not approve of / approve of with full knowledge, whether secretly (and whether illegally) or not. Examples of malware include Trojans, viruses, rootkits, spyware, hacking tools, keyloggers, etc. One example of malware is a desktop application that collects and reports the location of an end user to a remote server (but does not provide a location-based service, such as mapping service, to the user). Another example of malware is a malicious Android application package (.apk) file that appears to be a free game to an end user, but stealthily sends SMS premium messages (e.g., at $10 each), quickly accumulating the end user's phone bill. Another example of malware is an Apple iOS flashlight application that stealthily collects the user's contacts and sends those contacts to a spammer. Other forms of malware (e.g., ransomware) can also be detected / thwarted using the techniques described herein. Moreover, while feature vectors are described herein as being generated for detecting malicious JavaScript source code, the techniques described herein can also be used in various embodiments to generate feature vectors for other types of source code (e.g., HTML and / or other programming / scripting languages).

[0042] The techniques described herein can be used in conjunction with a variety of platforms (e.g., desktop, mobile device, gaming platform, embedded system, etc.) and / or a variety of types of applications (e.g., Android.apk files, iOS applications, Windows PE files, Adobe Acrobat PDF files, etc.). In particular, the techniques described herein can be used in conjunction with mobile applications (e.g., Android.apk files, iOS applications, etc.) and / or JavaScript source code.​Figure 1 In the example environment shown in FIG. 1, client devices 104-108 are laptop computers, desktop computers, and tablet computers, respectively, that are present in enterprise network 140. Client device 110 is a laptop computer that is present outside of enterprise network 140.

[0043] Data appliance 102 is configured to enforce policies related to communications between client devices, such as client devices 104 and 106, and nodes that are external to enterprise network 140 (e.g., reachable via external network 118). Examples of such policies include policies that govern traffic shaping, quality of service, and traffic routing. Other examples of policies include security policies, such as policies that require scanning for threats in incoming (and / or outgoing) email attachments, website content, files exchanged through instant messaging programs, and / or other file transfers. In some embodiments, data appliance 102 is also configured to enforce policies with respect to traffic that stays within enterprise network 140.

[0044] Figure 2A An embodiment of a data appliance is shown in FIG. 2. The example shown is a representation of the physical components included in data appliance 102 in various embodiments. Specifically, data appliance 102 includes a high-performance multi-core central processing unit (CPU) 202 and random access memory (RAM) 204. Data appliance 102 also includes storage 210, such as one or more hard disks or solid state storage units. In various embodiments, data appliance 102 stores (whether in RAM 204, storage 210, and / or other appropriate locations) information used in monitoring enterprise network 140 and implementing the disclosed technology. Examples of such information include application identifiers, content identifiers, user identifiers, requested URLs, IP address mappings, policies and other configuration information, signatures, hostname / URL classification information, malware profiles, and machine learning models. Data appliance 102 can also include one or more optional hardware accelerators. For example, data appliance 102 can include a crypto engine 206 configured to perform encryption and decryption operations, and one or more field programmable gate arrays (FPGAs) 208 configured to perform matching, act as a network processor, and / or perform other tasks.

[0045] The functionality described herein as being performed by the data appliance 102 can be provided / implemented in a variety of ways. For example, the data appliance 102 can be a dedicated device or set of devices. The functionality provided by the data appliance 102 can also be integrated into a general purpose computer, computer server, gateway, and / or network / routing device, or performed as software on a general purpose computer, computer server, gateway, and / or network / routing device. In some embodiments, at least some of the services described as being provided by the data appliance 102 are instead (or additionally) provided to a client device by software executing on the client device (e.g., client device 104 or client device 110).

[0046] Whenever the data appliance 102 is described as performing a task, the individual components, a subset of components, or all components of the data appliance 102 can cooperate to perform the task. Similarly, whenever a component of the data appliance 102 is described as performing a task, a sub-component can perform the task and / or the component can perform the task in conjunction with other components. In various embodiments, portions of the data appliance 102 are provided by one or more third parties. Depending on factors such as the amount of computing resources available to the data appliance 102, various logical components and / or features of the data appliance 102 can be omitted, and the techniques described herein adapted accordingly. Similarly, additional logical components / features can be included in embodiments of the data appliance 102, as applicable. In various embodiments, one example of a component included in the data appliance 102 is an application identification engine, configured to identify applications (e.g., using various application signatures, for identifying applications based on packet flow analysis). For example, the application identification engine can determine what type of traffic a session involves, such as web browsing - social networking; web browsing - news; SSH, and so on.

[0047] Figure 2B is a functional diagram of logical components of a data appliance embodiment. The example shown is a representation of logical components that can be included in the data appliance 102 in various embodiments. Unless otherwise specified, the various logical components of the data appliance 102 can be implemented in a variety of ways, including as a set of one or more scripts (e.g., written in Java, python, and so on, as applicable).

[0048] As shown, the data appliance 102 includes a firewall, and includes a management plane 232 and a data plane 234. The management plane is responsible for managing user interactions, such as by providing a user interface for configuring policies and viewing log data. The data plane is responsible for managing data, such as by performing packet processing and session handling.

[0049] The network processor 236 is configured to receive packets from a client device, such as the client device 108, and provide them to the data plane 234 for processing. Whenever the flow module 238 identifies a packet as part of a new session, it creates a new session flow. Based on the flow lookup, subsequent packets will be identified as belonging to the session. SSL decryption is applied by the SSL decryption engine 240, if applicable. Otherwise, the processing of the SSL decryption engine 240 is omitted. The decryption engine 240 can help the data appliance 102 inspect and control SSL / TLS and SSH encrypted traffic, and thereby help stop threats that might otherwise still be hidden in encrypted traffic. The decryption engine 240 can also help prevent sensitive content from leaving the enterprise network 140. Decryption can be selectively controlled (e.g., enabled or disabled) based on parameters such as URL category, traffic source, traffic destination, user, user group, and port. In addition to a decryption policy (e.g., a policy specifying which sessions to decrypt), a decryption profile can be assigned to control various options for sessions controlled by the policy. For example, a particular cipher suite and encryption protocol version can be required to be used.

[0050] The application identification (APP-ID) engine 242 is configured to determine what type of traffic a session involves. As one example, the application identification engine 242 can identify a GET request in received data and infer that the session requires an HTTP decoder. In some cases, such as a web browsing session, the identified application can change, and such changes will be recorded by the data appliance 102. For example, a user can initially browse to a company wiki (classified as "Web Browsing - Productivity" based on the URL visited) and then subsequently browse to a social networking site (classified as "Web Browsing - Social Networking" based on the URL visited). Different types of protocols have corresponding decoders.

[0051] Based on the determinations made by the application identification engine 242, the threat engine 244 sends the packets to the appropriate decoder, which is configured to assemble the packets (which can be received out of order) into the correct order, perform tokenization (e.g., tokenization is described further below), and extract out information. The threat engine 244 also performs signature matching to determine what should happen to the packet. As needed, the SSL encryption engine 246 can re-encrypt the decrypted data. The packets are forwarded for transmission (e.g., to a destination) using the forwarding module 248.

[0052] Also as Figure 2BAs shown in the middle, policies 252 are received and stored in the management plane 232. The policies can include one or more rules that can be specified using domain names and / or host / server names, and the rules can apply one or more signatures or other matching criteria or heuristics, such as for security policy enforcement of subscribers / IP flows based on various parameters / information extracted from monitored session traffic flows. An interface (I / F) communicator 250 is provided for managing communications (e.g., via (REST) API, messaging, or network protocol communications, or other communication mechanisms).

[0053] Example Security Platform

[0054] Returning to Figure 1 , assume that a malicious individual (using system 120) has created malware 130. The malicious individual wants a client device such as client device 104 to execute a copy of malware 130 that compromises the client device and, for example, causes the client device to become a bot in a botnet. The compromised client device can then be instructed to perform tasks (e.g., cryptocurrency mining, or participate in a denial-of-service attack), and report information to and receive instructions from external entities such as a command and control (C&C) server 150, as applicable.

[0055] Assume that data appliance 102 has intercepted an email sent to user "Alice" operating client device 104 (e.g., by system 120). A copy of malware 130 has been appended to the message by system 120. As an alternative but similar scenario, data appliance 102 can intercept an attempted download of malware 130 by client device 104 (e.g., from a website). In either scenario, data appliance 102 determines whether a signature of the file (e.g., the email attachment or website download of malware 130) exists on data appliance 102. The signature, if it exists, can indicate that the file is known to be safe (e.g., listed on a whitelist), and can also indicate that the file is known to be malicious (e.g., listed on a blacklist).

[0056] In various embodiments, the data appliance 102 is configured to work in cooperation with the security platform 122. As one example, the security platform 122 can provide the data appliance 102 with a set of signatures of known malicious files (e.g., as part of a subscription). If the signature of the malware 130 is included in the set (e.g., the MD5 hash of the malware 130), the data appliance 102 can prevent the transmission of the malware 130 to the client device 104 accordingly (e.g., by detecting that the MD5 hash of an email attachment sent to the client device 104 matches the MD5 hash of the malware 130). The security platform 122 can also provide the data appliance 102 with a list of known malicious domains and / or IP addresses, allowing the data appliance 102 to block traffic between the enterprise network 140 and the C&C server 150 (e.g., in the event that the C&C server 150 is known to be malicious). The list of malicious domains (and / or IP addresses) can also help the data appliance 102 determine when one of its nodes has been compromised. For example, if the client device 104 attempts to contact the C&C server 150, such an attempt is a strong indicator that the client 104 has been compromised by malware (and remedial action should be taken accordingly, such as quarantining the client device 104 from communicating with other nodes within the enterprise network 140). As will be described in greater detail below, the security platform 122 can also provide the data appliance 102 with other types of information (e.g., as part of a subscription), such as a set of machine learning models that can be used by the data appliance 102 to perform online analysis of files.

[0057] In various embodiments, if no signature is found for the attachment, the data appliance 102 can take a variety of actions. As a first example, the data appliance 102 can fail-safe by blocking the transmission of any attachment that is not whitelisted as benign (e.g., does not match a signature of a known good file). The downside of this approach is that there can be many legitimate attachments (when in fact they are benign) that are unnecessarily blocked as potential malware. As a second example, the data appliance 102 can fail-danger by allowing the transmission of any attachment that is not blacklisted as malicious (e.g., does not match a signature of a known bad file). The downside of this approach is that newly created malware (that the platform 122 has not seen before) will not be prevented from causing harm.

[0058] As a third example, the data appliance 102 can be configured to provide a file (e.g., malware 130) to the security platform 122 for static / dynamic analysis to determine whether it is malicious and / or to otherwise classify it. While the security platform 122 is analyzing the attachment (for which a signature does not yet exist), the data appliance 102 can take a variety of actions. As a first example, the data appliance 102 can prevent the email (and attachment) from being delivered to Alice until a response is received from the security platform 122. Assuming that the platform 122 takes approximately 15 minutes to thoroughly analyze the sample, this means that the incoming message to Alice will be delayed by 15 minutes. In this example, since the attachment is malicious, such a delay will not negatively impact Alice. In an alternative example, assume that someone sent a time-sensitive message to Alice with a benign attachment for which a signature also does not exist. Delaying delivery of the message to Alice by 15 minutes would likely be deemed unacceptable (e.g., by Alice). As will be described in greater detail below, an alternative approach is to perform at least some real-time analysis on the attachment on the data appliance 102 (e.g., while waiting for a ruling from the platform 122). If the data appliance 102 can independently determine whether the attachment is malicious or benign, it can take an initial action (e.g., block or allow delivery to Alice), and in applicable cases, adjust / take additional actions once a ruling is received from the security platform 122.

[0059] The security platform 122 stores a copy of the received sample in storage 142 and begins analysis (or schedules analysis, as applicable). One example of the storage 142 is an Apache Hadoop cluster (HDFS). Results of the analysis (as well as additional information related to the application) are stored in database 146. In the event that the application is determined to be malicious, the data appliance can be configured to automatically block file download based on the analysis results. In addition, a signature can be generated for the malware and the signature distributed (e.g., to data appliances such as 102, 136, and 148) to automatically block future file transfer requests to download files determined to be malicious.

[0060] In various embodiments, the security platform 122 comprises one or more specialized commercially available hardware servers (e.g., with multi-core processor(s), 32G+ of RAM, gigabit network interface adapter(s), and hard drive(s)) running a typical server class operating system (e.g., Linux). The security platform 122 can be implemented across a scalable infrastructure including multiple such servers, solid state drives, and / or other applicable high performance hardware. The security platform 122 can include several distributed components, including components provided by one or more third parties. For example, portions or all of the security platform 122 can be implemented using Amazon Elastic Compute Cloud (EC2) and / or Amazon Simple Storage Service (S3). Further, as with the data appliance 102, whenever the security platform 122 is mentioned as performing a task such as storing data or processing data, it should be understood that one sub-component or multiple sub-components of the security platform 122 (whether alone or in cooperation with third party components) can cooperate to perform the task. As one example, the security platform 122 can optionally cooperate with one or more virtual machine (VM) servers, such as the VM server 124, to perform static / dynamic analysis.

[0061] An example of a virtual machine server is a physical machine comprising commercially available server class hardware (e.g., multi-core processor, 32+ gigabytes of RAM, and one or more gigabit network interface adapters) running commercially available virtualization software such as VMware ESXi, Citrix XenServer, or Microsoft Hyper-V. In some embodiments, the virtual machine server is omitted. Further, the virtual machine server can be under the control of the same entity that administers the security platform 122, but can also be provided by a third party. As one example, the virtual machine server can rely on EC2, with the remainder of the security platform 122 being provided by specialized hardware owned by and under the control of the operator of the security platform 122. The VM server 124 is configured to provide one or more virtual machines 126-128 for emulating client devices. The virtual machines can execute a variety of operating systems and / or versions thereof. Observations of behavior resulting from execution of applications in the virtual machines (e.g., indications that an application is malicious) are logged and analyzed. In some embodiments, log analysis is performed by the VM server (e.g., the VM server 124). In other embodiments, the analysis is performed at least in part by other components of the security platform 122, such as the coordinator 144.

[0062] In various embodiments, the security platform 122 makes the results of its sample analysis available to the data appliance 102 as part of a subscription via a list of signatures (and / or other identifiers). For example, the security platform 122 can periodically send a content package identifying malware apps (e.g., every day, every hour, or some other interval, and / or based on events configured by one or more policies). An example content package includes a list of identified malware apps with information such as a package name, a hash value for uniquely identifying the app, and a malware name (and / or malware family name) for each identified malware app. The subscription can encompass analysis of only those files that are intercepted by the data appliance 102 and sent by the data appliance 102 to the security platform 122, and can also encompass signatures for all malware known to the security platform 122 (or a subset thereof, such as only mobile malware but not other forms of malware (e.g., PDF malware)). As will be described in more detail below, the platform 122 can also make other types of information available, such as machine learning models (e.g., based on feature vectors) that can assist the data appliance 102 in detecting malware (e.g., by techniques other than hash-based signature matching).

[0063] In various embodiments, the security platform 122 is configured to provide security services to a variety of entities in addition to (or, where applicable, instead of) the operator of the data appliance 102. For example, other enterprises with their own respective enterprise networks 114 and 116 and their own respective data appliances 136 and 148 can contract with the operator of the security platform 122. Other types of entities can also utilize the services of the security platform 122. For example, an Internet service provider (ISP) that provides Internet service to the client devices 110 can contract with the security platform 122 to analyze applications that the client devices 110 attempt to download. As another example, the owners of the client devices 110 can install software on the client devices 110 that communicates with the security platform 122 (e.g., to receive content packages from the security platform 122, use the received content packages to inspect attachments according to the techniques described herein, and transmit applications to the security platform 122 for analysis.

[0064] Analyzing samples using static / dynamic analysis

[0065] Figure 3FIGURE illustrates an example of logical components that can be included in a system for analyzing samples. The analysis system 300 can be implemented using a single device. For example, the functionality of the analysis system 300 can be implemented in the malware analysis module 112 incorporated into the data appliance 102. The analysis system 300 can also be collectively implemented across multiple different devices. For example, the functionality of the analysis system 300 can be provided by the security platform 122.

[0066] In various embodiments, the analysis system 300 utilizes a list, database, or other collection of known safe content and / or known bad content (collectively shown as the collection 314) in performing its analysis. The collection 314 can be obtained in a variety of ways, including via a subscription service (e.g., provided by a third party) and / or as a result of other processing (e.g., performed by the data appliance 102 and / or the security platform 122). Examples of information included in the collection 314 are: URLs, domain names, and / or IP addresses of known malicious servers; URLs, domain names, and / or IP addresses of known safe servers; URLs, domain names, and / or IP addresses of known command and control (C&C) domains; signatures, hashes, and / or other identifiers of known malicious applications; signatures, hashes, and / or other identifiers of known safe applications; signatures, hashes, and / or other identifiers of known malicious files (e.g., Android exploit files); signatures, hashes, and / or other identifiers of known safe libraries; and signatures, hashes, and / or other identifiers of known malicious libraries. Figure 3

[0067] ingestion

[0068] In various embodiments, when a new sample is received for analysis (e.g., an existing signature associated with the sample does not exist in the analysis system 300), it is added to the queue 302. As shown in Figure 3 In various embodiments, the malware 130 is received by the system 300 and added to the queue 302.

[0069] static analysis

[0070] The coordinator 304 monitors the queue 302 and, when resources (e.g., static analysis workers) become available, the coordinator 304 takes a sample from the queue 302 for processing (e.g., takes a copy of the malware 130). In particular, the coordinator 304 first provides the sample to the static analysis engine 306 for static analysis. In some embodiments, one or more static analysis engines are included within the analysis system 300, where the analysis system 300 is a single device. In other embodiments, static analysis is performed by a separate static analysis server that includes multiple workers (i.e., multiple instances of the static analysis engine 306).

[0071] ​The static analysis engine obtains general information about the sample and includes it (along with heuristics and other information, where applicable) in a static analysis report 308. The report can be created by the static analysis engine, or by the coordinator 304 (or by another appropriate component), which can be configured to receive information from the static analysis engine 306. In some embodiments, instead of or in addition to a separate static analysis report 308 being created (i.e., part of the database record forming the report 308), the collected information is stored in the database record for the sample (e.g., in the database 316). In some embodiments, the static analysis engine also forms a verdict about the application (e.g., "safe," "suspicious," or "malicious"). As one example, the verdict can be "malicious" if even one "malicious" static feature is present in the application (e.g., the application includes a hard link to a known malicious domain). As another example, points can be assigned to each feature (e.g., based on severity if found; based on how reliable the feature is for predicting maliciousness; etc.), and the verdict can be assigned by the static analysis engine 306 (or, where applicable, the coordinator 304) based on the number of points associated with the static analysis results.

[0072] Dynamic analysis

[0073] Once static analysis is complete, the coordinator 304 locates available dynamic analysis engines 310 to perform dynamic analysis on the application. As with the static analysis engine 306, the analysis system 300 can directly include one or more dynamic analysis engines. In other embodiments, dynamic analysis is performed by a separate dynamic analysis server that includes multiple workers (i.e., multiple instances of the dynamic analysis engine 310).

[0074] Each dynamic analysis worker manages a virtual machine instance. In some embodiments, the results of static analysis (e.g., performed by static analysis engine 306), whether in report form (308) and / or as stored in database 316, or otherwise, are provided as input to dynamic analysis engine 310. For example, static report information can be used to help select / customize the virtual machine instance used by dynamic analysis engine 310 (e.g., Microsoft Windows 7 SP 2 versus Microsoft Windows 10 Enterprise, or iOS 11.0 versus iOS 12.0). In cases where multiple virtual machine instances are executed concurrently, a single dynamic analysis engine can manage all instances, or multiple dynamic analysis engines can be used (e.g., with each dynamic analysis engine managing its own virtual machine instance), as applicable. As will be explained in greater detail below, during the dynamic portion of the analysis, the actions taken by the application are analyzed, including network activity.

[0075] In various embodiments, static analysis of a sample is omitted or performed by a separate entity, as applicable. As one example, traditional static and / or dynamic analysis of a file can be performed by a first entity. Once a given file is determined (e.g., by the first entity) to be malicious, the file can be provided to a second entity (e.g., an operator of security platform 122) that specializes in performing additional analysis regarding the use of network activity by malware (e.g., by dynamic analysis engine 310).

[0076] The environment used by analysis system 300 is instrumented / hooked such that behavior observed while an application is executing is logged as they occur (e.g., using a custom kernel that supports hooking and logcat). Network traffic associated with the emulator is also captured (e.g., using pcap). The logs / network data can be stored on analysis system 300 as temporary files, and can also be stored more permanently (e.g., using HDFS or another suitable storage technology or combination of technologies, such as MongoDB). The dynamic analysis engine (or another suitable component) can compare the connections made by the sample to a list of domains, IP addresses, etc. (314), and determine whether the sample has communicated with (or attempted to communicate with) a malicious entity.

[0077] Like the static analysis engine, the dynamic analysis engine stores the results of its analysis in the database 316 in a record associated with the application being tested (and / or includes the results in the report 312, as applicable). In some embodiments, the dynamic analysis engine also forms a verdict about the application (e.g., "safe," "suspicious," or "malicious"). As one example, the verdict can be "malicious" if the application takes even one "malicious" action (e.g., makes an attempt to contact a known malicious domain, or observes an attempt to leak sensitive information). As another example, points can be assigned to the actions taken (e.g., based on severity if found; based on how reliable the action is for predicting maliciousness; and so on), and the verdict can be assigned by the dynamic analysis engine 310 (or, as applicable, the coordinator 304) based on the number of points associated with the dynamic analysis results. In some embodiments, a final verdict associated with a sample is made based on a combination of the report 308 and the report 312 (e.g., by the coordinator 304).

[0078] Additional details regarding the threat engine

[0079] In various embodiments, the data appliance 102 includes a threat engine 244. The threat engine incorporates both protocol decoding and threat signature matching during respective decoder stages and pattern matching stages. The results of the two stages are merged by a detector stage.

[0080] When the data appliance 102 receives a packet, the data appliance 102 performs session matching to determine which session the packet belongs to (allowing the data appliance 102 to support concurrent sessions). Each session has a session state that involves a particular protocol decoder (e.g., a web browsing decoder, an FTP decoder, or an SMTP decoder). When a file is transmitted as part of a session, the applicable protocol decoder can utilize an appropriate file-specific decoder (e.g., a PE file decoder, a JavaScript decoder, or a PDF decoder).

[0081] Portions of an example embodiment of the threat engine 244 are described in Figure 4context (e.g., encountered when processing a JavaScript file). Decoder 402 can tag the end-of-file context in the packet, which can then be used to trigger execution of an appropriate model using observed features of the file. In some cases (e.g., FTP traffic), there can be no explicit protocol-level label for decoder 402 to identify / tag the context. In another embodiment, decoder component 402 is configured to determine a file type associated with each file in sample(s) 404 (e.g., a malware sample can include various source code content and / or other types of content for malware analysis, such as JS code, HTML code, and / or other programming / scripting languages, as well as other structured text such as URLs, or unstructured content such as images, etc.) and can decode the file to perform static analysis using IUPG model(s) as further described below. As will also be described in further detail below, in various embodiments, decoder 402 can use other information (e.g., file size reported as in a header) to determine when feature extraction of a file should end (e.g., a section begins) and when execution using an appropriate model should begin (e.g., as further described below, decoder 402 can determine a file type associated with sample(s) 404 and then select an appropriate IUPG model for that source code type associated with the file type, such as a JS IUPG model for JS files, an HTML IUPG model for HTML files, etc., and analyzer 406 can perform static analysis of the sample using the appropriate IUPG model(s)).

[0082] Threat engine 244 also includes an analyzer component 406 for performing static analysis of sample(s) 404 using the selected IUPG model(s), as further described below. Detector component 408 (e.g., using the target feature vectors of the selected IUPG model(s)) determines whether to classify each of the analyzed sample(s) 404 as malicious or benign (e.g., based on a threshold score), as will also be further described below. As one example, analyzer(s) 406 and detector 408 can be performed by data appliance 102 and / or by a security agent / software executing on client 110 (e.g., and also as described in further detail in U.S. Patent Application No. 62 / 729, 1 10, filed September 13, 2018, entitled "SYSTEM AND METHOD FOR MALWARE ANALYSIS," which is incorporated by reference in its entirety). Figure 1Similarly illustrated by the analyzer and detector 154 of the security platform 122) using the disclosed techniques for IUPG models applied to malware classification based on static analysis of source code samples. The detector 408 processes the output provided by the decoder 402 and analyzer(s) 406 to take various responsive actions (e.g., based on security policy / rule(s)).

[0083] Innocent until proven guilty (IUPG) introduction: building deep learning models that are both adversary- and false-positive-resistant

[0084] Classification cross-entropy (CCE) loss is a standard supervised loss function used to train a variety of deep neural network (DNN) classifiers. CCE produces a purely discriminative model that has no means to infer out-of-distribution (OOD) content or effectively utilize classes that do not possess uniquely identifiable structure. The disclosed techniques for providing an innocent until proven guilty (IUPG) framework provide alternative architecture components and hybrid discriminative and generative loss functions for training DNNs to classify mutually exclusive classes. IUPG includes learning a library of inputs within the original input space that, together with the network, constitute prototypes of uniquely identifiable subsets in the input space. The network learns to map the input space to an output vector space in which the prototypes and members of related input subsets are mapped exclusively to common points in the output vector space. Noise (or any class of data that lacks a prototype description) and the distance between all prototypes in the output vector space are maximized in training. When a target class has one or more designated prototypes, we call any such class a "deviated target." Deviated target data helps to pare down the extracted features of a target class to those that are truly class-unique as opposed to coincidental.

[0085] For example, machine learning techniques (MLT) as applied in computer / network security have significant challenges - it generally should not make mistakes. A mistake in one direction can lead to a dangerous misstep of letting malware fall into a crack (e.g., malware is allowed to penetrate a corporate network or execute on a computing entity such as a server, computing endpoint, etc.). A mistake in the other direction causes your security solution to block, for example, benign traffic or executables on computing entities, which is also costly for a network security company and a huge nuisance for users. Generally, the amount of good (benign) traffic vastly exceeds the amount of malicious traffic, so it is generally desirable for an effective and efficient security solution to minimize the amount of good traffic that is called malicious (e.g., generally known as false positives (FP)). Malware authors understand this and try to disguise their malicious code to look more like benign code. The most direct way to accomplish this is generally known as an append attack (e.g., also known as injection, bundling, etc.), where an attacker takes (e.g., typically a large amount of) some amount of benign content and injects their malicious content into it without compromising the functionality of the malware, such as will be described further below. Because a machine learning (ML) classifier built with standard techniques is sensitive to the presence of it, a significant amount of benign content can interfere with the classification decision, causing it to deviate from a positive malware decision, thus sometimes causing the classifier to completely miss it.

[0086] In the context of malware classification using IUPG techniques, this is equivalent to learning inseparable features of a malware cluster that define its maliciousness while ignoring benign content. In an example implementation, during inference, each sample is scanned for these inseparable qualities while ignoring all structure outside of this class, so the technique assumes that each sample is "innocent until proven guilty." Increasing the specificity of the learned features intuitively increases the network's resistance to any OOD content (e.g., noise resistance as described further below). As a central assumption of the disclosed IUPG techniques, we posit that this increased resistance to any OOD content is a primary cause of the desirable effects we explore as described further below.

[0087] The disclosed IUPG techniques accomplish this by learning an abstract input library within the raw representation of the data that, together with the layer operations of the network, constitutes a prototype of the input space subset. The network learns to map the input space to an output vector space where the prototype and members of the relevant input subset are specifically mapped to a common point. The distance between noise (or any class of data that lacks a prototype description, which we will generally call "off-target") and all prototype inputs in the output vector space is maximized in training. Thus, off-target data helps to whittle down the extracted features of the target class to be truly unique to the class as opposed to coincidental.

[0088] Increasing the specificity of the learned representation of the class (while still balancing the ubiquity in the loss) naturally increases the network's resistance to input noise. We hypothesize that the embedded noise-robust properties of the IUPG network are the primary reason for the desirable qualities we explore in this work. We measure baseline performance using an equivalent network topology trained with CCE loss. We refer to this control network setup as the CCE counterpart of the IUPG network. In our evaluations, we (1) explore the test set classification performance of the IUPG and its CCE counterpart across various network security and computer vision experimental setups that include different uses of noise; (2) measure the tendency of both frameworks to produce false positive (FP) responses to OOD inputs; (3) measure the resistance of both frameworks to black-box adversarial attacks using both standard training and our introduced custom adversarial training procedure; and (4) demonstrate the applicability of existing adversarial learning techniques to the IUPG.

[0089] As will be described below with respect to various embodiments, by increasing the specificity of structured class models via prototype-based learning and unique handling of unstructured classes, IUPG trained networks can provide significant advantages in common real-world problem settings, such as certain noise-based adversarial attacks, handling distribution shifts, and out-of-distribution classification in computer / network security contexts. Because the IUPG is general enough to be applied to any architecture where categorical cross-entropy (CCE) can be used, there are various opportunities for the IUPG to be combined with existing adversarial learning / OOD detectors that present better performance than either technique used in isolation, as will also be further described below.

[0090] We show that the unique benefits of the IUPG are particularly useful for malware classification efforts. In the context of malware classification, adversarial attacks can lead to risky false negatives, while OOD failures can lead to costly false positives. In sum, the various novel aspects disclosed herein include the following without limitation: (1) proposing the IUPG framework; (2) demonstrating several benefits of using the IUPG on CCE loss discussed above; (3) proposing novel architectures and training procedures to build adversarial attack-resistant DNN malware classifiers; and (4) applying the IUPG framework to effectively and efficiently detect various forms of malware (e.g., JavaScript-related malware, URL-related malware, and / or other forms of malware can be similarly classified and detected using the disclosed IUPG techniques and IUPG framework, as will be further described below).

[0091] As will be further described below, experimental results reveal that the additional attacks can be significantly successful even for highly accurate classifiers. As an example, for a deep learning JavaScript (JS) malware classifier that we constructed with a classification cross-entropy (CCE) loss, despite the classifier achieving >99% accuracy on its test set, it took only 10,000 random benign content characters to append to a malicious sample to successfully flip the verdict at >50% of the time. This is particularly worrisome given the extremely low cost of the exploitation attack. The adversary does not need to know any details about the victim classifier, while at the same time benign content is extremely abundant and simple to produce. If the adversary has access to sensitive information about the victim model, such as its loss function, then the appended content can be designed with model-specific techniques, which generally further increase the success rate.

[0092] To address this technically challenging problem, it is not exclusively indicated that the content of malware should generally have a small enough impact on the classification mechanism such that a verdict will not be flipped to benign. At a high level, our approach is to encourage the network to specialize in learning and recognizing uniquely identifiable patterns of the malicious class, while being explicitly robust to all other content. An important observation is that malware patterns are highly structured and uniquely identifiable compared to the infinite possibilities of benign patterns that you can encounter in data. As such, an example innovation of the disclosed IUPG techniques is to distinguish between classes that do and do not have uniquely identifiable structures (e.g., patterns) in their use of learning. In the malware classification context, the malware class generally has uniquely identifiable structures (e.g., referred to herein as the target class), while the benign class is inherently random (e.g., referred to herein as the off-target class). As further described below, the disclosed IUPG techniques are specifically designed to learn the uniquely identifiable structures within the target class, while only utilizing the off-target class to whittle down the representation of the target class to what is truly inseparable. This facilitates reducing the overall receptive field of the neural network - i.e., the data patterns it is sensitive to - to specifically reduce to the malicious patterns that are clear indicators of a correct positive verdict. If no such malicious patterns are found, then and only then does a benign verdict arise. That is, unknown files are innocent until proven guilty (IUPG). This is in contrast to regular unbound learning, which generally freely identifies benign patterns that can help minimize loss, but ultimately do not impart any information about file safety as a whole. Furthermore, we assume that any benign patterns learned by the classifier are likely to be nothing more than overfitting characteristics to aspects of the environment training data. Since there are almost infinite possibilities of expression, it is at best inefficient to try to capture benign patterns. In the worst case, it leads to overfitting characteristics, which opens up the classifier to susceptibility to additional attacks. Worse still, if your training, validation, and test splits draw from the same distribution (e.g., which is common practice), then standard test set classification metrics will likely fail to elucidate the problem, as the benign characteristics of the classifier can still lead to good performance against the test set. Only when you put the classifier on real-world (e.g., important place) data outside of your training distribution will you incur troublesome classification errors and attack vulnerabilities.

[0093] As will be described in more detail below, in the evaluations, we measure baseline performance using an equivalent network trained with CCE loss - the CCE counterpart of the IUPG network in this paper. We (1) explore test set classification performance of IUPG and its CCE counterpart across various network security and computer vision settings including different uses of noise; (2) compare the tendency to produce false positive (FP) responses to OOD inputs; (3) compare the effect of recent bias (e.g., performance loss due to distribution shift) on classification accuracy; (4) compare the resistance of both frameworks to black-box adversarial attacks; and (5) demonstrate the applicability of existing adversarial training techniques to IUPG.

[0094] Background of Related Work

[0095] We summarize three key themes of related work, which include: (1) prototype-based learning, (2) adversarial attacks, and (3) out-of-distribution (OOD) attacks.

[0096] Prototype-based learning: Among the earliest work in prototype-based learning is learning vector quantization (LVQ) (see, e.g., T. Kohonen, “The self-organizing map,” Neurocomputing, 21(1): 1 - 6, 1998, ISSN 0925-2312, doi: https: / / doi.org / 10.1016 / S0925-2312(98)00030-7), which can be thought of as a prototype-based k-nearest neighbor algorithm. In taxonomy of LVQ variants presented in D. Nova and P. A. Estévez, “A review of learning vector quantization classifiers,” Neural Comput. Appl., 25(3-4):511-524, Sept. 2014. ISSN 0941-0643. doi: 10.1007 / s00521-013-1535-3), this work can be compared to GLVQ (see, e.g., A. Sato and K. Yamada, “Generalized learning vector quantization,” In Proceedings of the 8th International Conference on Neural Information Processing Systems, NIPS’95, page 423-429, Cambridge, MA, USA, 1995, MIT Press), which falls under the umbrella of margin maximization for data spaces with Euclidean distance. However, IUPG combines prototype learning with DNNs and uniquely uses deviant target samples.Common goals of prototype-based learning in DNNs include few-shot learning (see, e.g., X. Liu, et al., “Meta-learning based prototype-relation network for few-shot classification,” Neurocomputing, 383:224-234, 2020, ISSN 0925-2312, doi: https: / / doi.org / 10.1016 / j.neucom.2019.12.034) and adjudicating explainability (see, e.g., O. Li, H. Liu, C. Chen, and C. Rudin, “Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions,” 2018). The model in H.-M. Yang, X.-Y. Zhang, F. Yin, and C.-L. Liu, “Robust classification with convolutional prototype learning,” 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition, June 2018, doi: 10.1109 / cvpr.2018.00366 (Yang et al.) shares several similarities with IUPG. However, the prototypes in Yang et al. are defined in the output vector space of the model. These prototypes are not human-interpretable and do not have an intuitive initialization. Crucially, when prototypes are defined in this way, we observe frequent convergence to solutions where multiple prototypes merge to a common point. This is supported by the results in Yang et al., which report similar or worse performance compared to multiple prototypes per class. We can also not find similar work that leverages the deviation from target samples as IUPG does. This ability allows us to find considerable benefits on problems such as malware classification, where only one class has a uniquely identifiable structure.

[0097] Additional attacks: Additional attacks are intended to perturb the classification outcome by concatenating adversarial content to the input (see, e.g., R. R. Wiyatno, A. Xu, O. Dia, and A. de Berker, “Adversarial examples in modern machine learning: A review” (2019) (Wiyatno et al.)). This is particularly relevant for malware classification, where benign noise can be appended to malware to fool the classifier, although the malicious activity remains intact. Conversely, malicious content can be injected into large benign files to evade detection (e.g., benign library injection or also by adding more whitespace space is a common form of additional attack, such as using various jQuery plugins and custom bundled files with website dependencies, which can cause incorrect benign classification by many classifiers, as the classification outcome can be perturbed by such benign libraries / injection of other content into such files). In so-called white-box attacks (see, e.g., Wiyatno et al.), additional noise can be crafted with the exploitation of model details. In general, black-box adversarial attacks assume no knowledge of the model, and are often the only possible attack against proprietary defenses. This work provides evidence that even the simplest additional attack variants can pose a serious threat to highly accurate models. To our knowledge, previous work in deep learning lacks a general solution to additional attacks on malware.

[0098] Out-of-distribution (OOD) classification: It is well understood that DNNs trained with CCE (preserving some specialized highly nonlinear options such as RBF networks) are prone to produce highly overconfident posterior distributions for OOD inputs (see, e.g., V. Sehwag et al., “Analyzing the robustness of open world machine learning,” pages 105-116, Nov. 2019, ISBN 978-1-4503-6833-9, doi: 10.1145 / 3338501.3357372). Reliably handling OOD content is a key requirement for real-world systems. In orthogonal to our work, open world frameworks typically equip models with an external detector that aims to identify and discard OOD inputs (see, e.g., J. Chen, Y. Li, X. Wu, Y. Liang, and S. Jha, “Robust out-of-distribution detection for neural networks,” 2020). Other work relies on learning an external rejection function concurrently or after the classification network is trained (see, e.g., Y. Geifman and R. El-Yaniv, “Selective classification for deep neural networks,” in I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 4878-4887, Curran Associates, Inc., 2017). This work demonstrates the embedded proficiency capability to handle OOD content produced by IUPG alone.

[0099] Combination strengths: In general, any techniques developed in recent years to overcome white-box attacks for CCE-trained DNNs, such as the diverse array of adversarial training (see, e.g., Wiyatno et al.), can be equivalently applied to IUPG-trained networks. Within these special training procedures, the IUPG loss can be used as a temporary replacement for CCE. Examples of this are further provided below, where we find that the success rate of IUPG is consistently higher. Similarly, we propose that a combination of IUPG with an external OOD detector is likely to outperform either in isolation, as will now be further described below with respect to various embodiments.

[0100] Innocent until proven guilty (IUPG) technology overview: building and using adversarial and false positive resistant deep learning models

[0101] Disclosed are techniques for providing innocent until proven guilty (IUPG) solutions for building and using adversarial and false positive resistant deep learning models. In some embodiments, a system, process, and / or computer program product includes storing a collection comprising one or more innocent until proven guilty (IUPG) models for static analysis of a sample; performing static analysis of content associated with the sample, wherein performing the static analysis includes using at least one stored IUPG model; and determining, based at least in part on the static analysis of the content associated with the sample, that the sample is malicious, and in response to determining that the sample is malicious, performing an action based on a security policy.

[0102] Deep neural network classifiers trained with conventional categorical cross-entropy loss face problems in real-world settings, such as a tendency to produce overconfident posterior distributions on out-of-distribution inputs, sensitivity to adversarial noise, and performance loss due to distribution shift. We hypothesize that the central shortcoming— the inability to effectively handle out-of-distribution content within inputs— exacerbates each of these setbacks. In response, we propose a novel learning framework, called innocent until proven guilty, that builds prototypes of training data clusters or classes within the input space while uniquely leveraging noise and inherent stochastic classes to discover adversarial, uniquely identifiable features of modeled classes. In evaluation, we utilize both academic computer vision datasets and real-world JavaScript and URL datasets for malware classification.

[0103] Across these interdisciplinary settings, we observe favorable classification performance on test data, reduced performance loss due to nearsightedness, reduced false positive responses to noisy samples, and reduced vulnerability in several noise-based attack simulations when compared to baseline networks trained with categorical cross-entropy.

[0104] The disclosed IUPG framework demonstrates a significant reduction in vulnerability to black-box additive attacks on malware. For example, by applying the well-known fast gradient sign method, we show the potential of combining our framework with existing adversarial learning techniques and discover a significant margin of favorable performance. Our framework is general enough to be used with any network topology that could otherwise be trained with categorical cross-entropy (CCE).

[0105] Example system embodiments of the framework

[0106] Figure 5A is an abstract network in accordance with some embodiments Block diagram of an enhanced IUPG assembly. Generally, an IUPG network is an encoder that maps inputs and prototypes from a common input space to an output vector space.

[0107] As a brief overview of the IUPG framework and IUPG technology, input samples and a library of prototypes are processed by the network in a Siamese fashion at each forward pass. These prototypes exist in the same input space as regular data points. For example, if you are classifying 28x28 images, each prototype will exist as a 28x28 matrix of learnable weights. In example implementations, there are two ways in which the prototypes can be defined: (1) directly as learnable members of the input space; or (2) more generally, as weights of a linear combination of a basis set of training data points. The latter is especially more convenient when the input space is excessively large. The samples and prototypes are mapped to a final output vector space, where a specially learned distance metric is paired with it. The IUPG learns the prototypes, network weights, and distance metric such that the output vector space orients all inputs, such as Figure 5B as shown in FIG. 5, using an IUPG network 500, as will be described below with reference to Figure 5A further.

[0108] In an ideal mapping, structured class members and their assigned prototype(s) map uniquely to a common point with some spatial margin, such that any possible input that is not a structured class member maps somewhere else. It should now be clear that a decision is made by measuring the distance of a mapped input to all mapped prototypes in this output vector space. If a mapped sample is measured to be close enough to a prototype, then it is predicted to be a member of the class to which that prototype is assigned. As shown in the space at 530, a noisy background (also referred to herein as off-target data) helps to elucidate (and capture in the prototypes) the true inseparability about a target class, such as class 1 prototype 532 and class 2 prototype 534 as shown in FIG. 5. Note that the IUPG network 500 can still be trained just fine without any off-target data or classes. We report stable or increased classification performance on several public datasets of this type. However, certain problems, such as malware classification, naturally fit with the ability to identify off-target data. Figure 5B

[0109] The IUPG loss encourages this ideal mapping by choreographing pushes and pulls between the sample and each prototype in the output vector space. The force exerted on each anchor sample is determined based on its label. Figure 5C The expected forces for on-target and off-target samples are illustrated. Note that for the off-target sample 550, it is pushed away from the anchor sample 540, which includes a prototype for the same class as the sample 550. The force exerted on the anchor sample 540 is determined based on its label, which is the same class as the sample 550. The force exerted on the sample 550 is determined based on its label, which is a different class than the anchor sample 540. Figure 5C ​Each of the target 1 prototype 552, target 2 prototype 554, and target 3 prototype 556 shown in the middle. This is achieved by using a zero vector as its one-hot label vector.

[0110] Returning to binary malware classification, as mentioned previously, we specify several prototypes for the malicious class while defining the benign class as deviating targets. It is now of interest to understand why it is necessary to learn unique identification patterns of malware while simultaneously robustly encoding benign content. In an ideal case, the prototypes and mappings of the network specifically capture inseparable features of malware families such that their activation is as strong an indication of malware as possible and no other features cause significant activation. This puts the adversary in a situation where the only way to disrupt the mappings to malicious prototypes is to twist or remove malware that actually does something malicious. With little activation on patterns that do not directly impart maliciousness, exploiting additional benign content will not help the adversary bypass the malware classifier. It is important to note that the tight and robust malware family clusters formed around prototypes in the output vector space while balancing with the universality of the loss such that, for example, isolated malware can still be reliably captured. In our experiments, in general, prototypes do not map to a single malware family or malicious pattern as if the model is reduced to simple pattern matching. Instead, the network and prototypes learn to recognize complex, high-level combinations of patterns that generalize across malware families while still remaining robust to benign activation.

[0111] Reference is now made to Figure 5A , the novel components of the IUPG framework 500 include the input and output layers of the DNN and a special loss function, as will be described below with reference to Figure 5A . All hidden layer details - including the number of layers of different functional types organized into any topology - can vary on demand based on the relevance of the problem. Consider a network that maps a vectorized input set to a vector in . The regular CCE training of maps to a vector in c (where is the number of classes). We will explain these example novel components of the IUPG framework by augmenting them onto the example abstract network architecture Figure 5A , as will now be described below with reference to

[0112] Data guide

[0113] For CCE training with c classes, all cLabel examples for a class are generally necessary and sufficient. While the IUPG can be trained with these data sets, we find it often useful to include off-target samples. For off-target samples we define How to determine what data or classes are “off-target” depends on the problem, but is intuitive. If training a “cat” or “non-cat” classifier, there is only one class with a uniquely identifiable structure. “Non-cat” does not possess a prototypical description. Any learned “non-cat” indicator is likely to be an aspect of the environment training data. “Non-cat” should be defined as an off-target class, while “cat” is assigned one or more prototypes. Alternatively, if training a “cat” or “dog” classifier, both classes possess uniquely identifiable structures, and off-target classes should be augmented to these classes. Statistical noise is often easily synthesized for off-target data.

[0114] Prototypes

[0115] The IUPG network processes input and a library of prototypes in a Siamese fashion (see, e.g., J. Bromley et al., “Signature verification using a “siamese” time delay neural network,” International Journal of Pattern Recognition and Artificial Intelligence, 7:25, Aug. 1993, doi: 10.1142 / S0218001493000339). P depicted as Figure 5A adjacent in 506. Each constitutes a learnable weight of the network. Each learns prototype information about a subset of the training data, such that all members of the subset will be exclusively mapped close to after processing with . The subset can be learned automatically, assigned with class labels, or both. One way to define each is as an element of itself, as shown in 502. If the elements of are excessively large, or large are desired, a memory-efficient definition of is as a weight vector that is a linear combination of the training inputs. Specifically, we specify static training samples to form the basis setB . B The elements of X are selected by a clustering technique to span the training distribution. We then define each Before processing with we compute the dot product Under both prototype definition variants, the selection of can be guided by domain knowledge or discovered through hyperparameter optimization techniques. Domain knowledge can guide the initialization of each For example, one might want to establish the cluster center points as an initialization that can correspond to semantically meaningful partitions of the classes. We found that clustering-based initialization significantly reduces the training time required for convergence.

[0116] The distance function

[0117] The vectors in 508 are mapped to the output vector space 510 via fully connected layers as shown in 512 and 514. k need not equal the number of classes. The intermediate representation of is denoted as while the representation of is denoted as as depicted in 510 in Figure 5. It is crucial to measure the distance between and each There are many options available to define the distance P (see, for example, S. Ontanon, “An overview of distance and similarity functions for structured data,” Artificial Intelligence Review, Feb 2020, ISSN 1573-7462, doi: 10.1007 / s10462-020-09821-w). We define the function This is followed by scaling with to ensure non-negativity after which the learned weight vector 516 is applied to the distance in each dimension. This function provides satisfactory results such that we do not feel the need to explore more options. As shown in 518, we use an adjusted sigmoidal function to bound all distances between Note that the hyperbolic tangent function can also achieve this. For input ​As shown at 520, the final vector defining the distance is made by setting a threshold on the value in .

[0118] The loss function

[0119] The IUPG loss seeks to minimize the distance of a sample to its designated prototype in while simultaneously maximizing the distance of the sample to all of its non-designated prototypes. The loss function presented in Equation 1 minimizes the sum of cross-entropy calculations between the label distribution of each target class and the prototype with the minimum distance in . We will define the loss for a single sample , where one-hot encodes c target classes. Assume that each of the target classes has designated prototypes. The generalized loss function for Figure 5A is shown in Equation 1, which is depicted in 526 in .

[0120]

[0121] When , we use 522 to measure the relative impact of the distance to the prototype designated as the class i . The sample recall for the off-target , which makes terms necessary, otherwise their loss would always be 0. Conceptually, when , we penalize the distance between and the closest prototype of the class i . When , we do the same for the inverse distance. We add a constant D to to avoid computing .

[0122] The assignment of prototypes to c target classes is specified inside the vector. The denoted prototype is designated as the target class . As shown at 524, we define each with in the case of and otherwise. Thus for the class​i Linear shift of the values so that the distance to the designated prototype is strictly larger than the distance to all other prototypes . Then compute gives us the minimum distance between the designed to class i prototypes.

[0123] Training and inference complexity

[0124] If all weights do not change, then only needs to be computed once and can then be reused. From the time complexity of the mapping is which is equal to its CCE counterpart when The continued computation of consists of the application of the dot product, and the application of which are scaled accordingly . Assuming is pre-computed, the previous two operations envelope the different operations of IUPG with respect to its CCE counterpart during inference. Note that both are highly parallelized and generally insignificant compared to the computation of During training, we additionally re-compute once per training batch. This is equivalent to adding samples to each batch. Note that IUPG also increases the number of learnable weights by .

[0125] Experiments

[0126] In our experiments, we consider malicious JavaScript (JS) and URL classification as well as MNIST (see, e.g., Yann LeCun and Corinna Cortes, “MNIST handwritten digit database” (2010)) and Fashion MNIST (see, e.g., Han Xiao, Kashif Rasul, and Roland Vollgraf, “Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms” (2017)) classification. For JS, we consider both the binary general malware classification problem and the multi-class malware family labeling problem. All models are implemented in TensorFlow (see, e.g., M. Abadi et al., “TensorFlow: Large-scale machine learning on heterogeneous distributed systems” (2015); and the TensorFlow open source software is available from tensorflow.org) and trained with the Adam optimizer (see, e.g., Diederik P. Kingma and Jimmy Ba, “Adam: A method for stochastic optimization” (2014)). A training batch size of 32 and a learning rate of 5 x 10 -5 are used throughout. We use ReLU and sigmoidal activations across all convolutional and fully connected layers, respectively. These hyperparameters allow both IUPG and CCE trained networks to converge after approximately the same number of batches. The shared hyperparameters used in this work were tuned when using the CCE loss—thus favoring the CCE counterpart. For IUPG, we set throughout. When defining all , we use K-means++ on the training data to compute prototype initialization (see, e.g., David Arthur and Sergei Vassilvitskii, “K-means++: The advantages of careful seeding” (in Proc. of the 18th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2007, pp. 1027-1035). Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2007, pages 1027-1035, USA, 2007, Society for Industrial and Applied Mathematics, ISBN9780898716245). When defining all at once, we use K-means++ instead to determine the members of B For our experiments, three different networks are used instead of . The topologies of all settings are illustrated in Figure 6 .

[0127] Figure 6 Figures illustrating the functions for (A) MNIST and Fashion MNIST, (B) JS and (C) URL according to some embodiments. For (A), the model comprises parallel convolutional layers. C:128 @5x5 is a convolutional layer of 128 5x5 filters. maxP@2x2 is a 2x2 max-pooling. FCC is a fully connected convolutional layer. FC:512 is a fully connected layer with 512 units. For (B), character-level and token-level input representations are processed independently. EVL refers to an embedded vector lookup operation. seqComp refers to a sequence compression operation. globalmaxP refers to a global max-pooling operation. For (C), C:128@11,3x30 indicates two different heights used in the filter groups: 11 for character-level input and 3 for token-level input.

[0128] Image classification

[0129] For brevity, MNIST (see, e.g., Yann LeCun and Corinna Cortes, “MNIST handwritten digit database” (2010)) and Fashion MNIST (see, e.g., Han Xiao, Kashif Rasul, and Roland Vollgraf, “Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms” (2017)) are considered roughly the same. Both datasets are split into random 50k-10k-10k Train-Test-Val (TTV) divisions. When needed, we generate Gaussian noise images and random stroke images with a random forest classifier to filter out accidental true positives. The images are preprocessed with max-min scaling and mean subtraction. When using IUPG, each Furthermore, we specify one prototype for each target class.

[0130] Malicious JS and static URL classification

[0131] Classifying malicious JavaScript and static URLs is a challenging task in web security (see, for example, "Jstap: A static pre-filter for malicious javascript detection" by Aurore Fass, Michael Backes, and Ben Stock, published in...). Proceedings of the 35th Annual Computer Security Applications Conference , ACSAC 2019, pages 257-269, New York, NY, USA, 2019, Association for ComputingMachinery); Yann LeCun and Corinna Cortes, “MNIST handwritten digit database” (2010); Doyen Sahoo, Chenghao Liu, and Steven C. H. Hoi, “Malicious URL detection using machine learning: A survey” (2017). Attachment attacks are particularly prevalent in JS malware, such as malicious injections into benign scripts. A simple yet crucial observation for malware classification is that benignity is only definable in the non-malicious context. For IUPG, we define benign data as deviating from the target class, that is, not modeled using a prototype.

[0132] Benign JavaScript is collected by crawling the top 1M domains from the Tranco list (see, for example, "Rigging research results by manipulating top websites rankings" by Victor LePochat, Tom van Goethem, and Wouter Joosen). CoRR (abs / 1806.01156, 2018). In addition to Tranco's filtering, we also ignored samples marked by state-of-the-art commercial URL filtering services. We utilized VirusTotal (VT) (see, for example, Gaurav Sood's "..."). virustotal: R Client for the virustotal API(2017, Rpackage version 0.2.1)) serves as the primary source of malicious JS samples. We require a minimum VT score (VTS) of three, which empirically demonstrates to be quite accurate. Our malicious and benign URL data is collected from internet traffic and external data sources (such as VT) using static and dynamic URL filters and analyzers from industrial cybersecurity companies. For binary JS malware classification, we use a 450k-600k-600k TTV split, where benign to malicious ratios are 70:30, 96:4, and 96:4, respectively. A large number of benign samples are included to accurately measure performance under strict false positive rate (FPR) requirements. Due to the high cost of FP, therefore FPR is common in industrial cybersecurity environments (see, for example, “The cost of malware containment,” produced by Byte Productions and maintained at www.byte-productions.com, January 2015). To build a multi-class malware family tagging classifier, we isolated nine different malware families, each with 10k-1k-1k TTV samples. Benign data with equal portions were added to form our multi-class training dataset. To generate OOD samples, we uniformly shuffled the token order in the benign scripts. For URLs, we used a 14M-2M-2M TTV split with a 50:50 class ratio. After the initial year of collection, we also collected a separate 2M, 50:50 test set to test recent biases.

[0133] exist Figure 6 of( B )and( C The architecture of our JS and URL classifier, illustrated in the diagram, is built upon various pre-existing works in NLP (e.g., see "Character-level convolutional networks for text classification" by Xiang Zhang, Junbo Zhao, and Yann LeCun, published in [Journal Name]). Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 , NIPS 2015, pages 649-657, Cambridge, MA, USA, 2015, MIT Press; Yoon Kim's "Convolutional neural networks for sentence classification" (published in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1746–1751, 2014); Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk's "Deep learning similarities from different representations of source code" (published in Proceedings of the 15th International Conference on Mining Software Repositories , MSR 2018, pages 542-553, New York, NY, USA, 2018, Association for Computing Machinery); Jack W. Stokes, Rakshit Agrawal, Geoff McDonald, and Matthew J. Hausknecht's "Scriptnet: Neural static analysis for malicious javascript detection" (published in 2019 IEEE Military Communications Conference, MILCOM 2019, Norfolk, VA, USA, November12-14, 2019 , pages 1–8, IEEE, 2019); Rie Johnson and Tong Zhang's "Semi-supervised convolutional neural networks for text categorization via region embedding" (published in C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28 , pages 919–927, Curran Associates, Inc., 2015); Rie Johnson and Tong Zhang's "Effective use of word order for text categorization with convolutional neural networks" (published in Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 103–112, Denver, Colorado, May–June 2015, Association for Computational Linguistics); “Adeep learning approach for detecting malicious javascript code” by Yao Wang, Wan-dong Cai, and Peng-cheng Wei ( Sec. and Commun. Netw. , 9(11):1520-1534, July 2016) and Hung Le, QuangPham, Doyen Sahoo and Steven CH Hoi, “Urlnet: Learning a URL representation with deep learning for malicious URL detection” (2018) . All inputs are represented at two levels of abstraction: characters and token streams. All URLs are populated to a fixed maximum size, while JS files are dynamically populated in batches. For the token-level representation, ( B A single-channel vocabulary using frequency-selected learning token embedded vectors. For ( C We include a char-by-word channel, similar to those in Hung Le, Quang Pham, DoyenSahoo, and Steven CH Hoi's "Urlnet: Learning a URL representation with deep learning for malicious URL detection" (2018). We additionally use independently trained Hidden Markov Models to generate a randomness score for each token, which scales the learned embedding vectors to generate a third randomness channel. When using IUPG, for JS, each... It has a fixed size. We specify one prototype for each family in the multi-class model, while the binary model has four specified prototypes for malicious JS classes—selected empirically. For the URL, we utilize both those defined as... All members Experiments were then conducted using a base set of 100 malicious URLs. Based on experience, four and 100 prototypes were specified for the malicious URL class, respectively.

[0134] Classification performance

[0135] On our various datasets, we explore classification performance with different combinations of training and testing with noise. When training the CCE counterpart with synthetic noise, we augment the dedicated noise class. For multi-class models, we define FP as the off-target samples classified as any target class. We use a single confidence threshold for all target classes. If exceeded, the maximum confidence target class is predicted. When testing without noise in Table 2, the maximum confidence target class is always predicted. Also note that in all tables except Table 2, the decision threshold is configured to obey the maximum FPR. Where applicable, results are presented over five trials with different random seeds. We find reliable stability or reductions in false negative rate (FNR), error percentage, and variants across Tables 1, 2, 3, and 4, as further described below. Figure 7B

[0136] Figure 7A Table 1 is illustrated, which includes malicious JS classification test set over all non-benign classes FNR. For multi-class models, the few-sample training dataset consists of 10 samples randomly selected per non-benign class; for binary models, 1000 randomly selected malware samples.

[0137] Figure 7B Table 2 is illustrated, which includes image classification noiseless test set Error percentage.

[0138] Figure 7C Table 3 is illustrated, which includes image classification test set when test set contains Gaussian noisy images and model is trained without noise Error percentage. Decision threshold is configured to obey the maximum FPR.

[0139] Figure 7D Table 4 is illustrated, which includes malicious URL classification test set FNR. Indicates that base set was used to define all Decision threshold is configured to obey the maximum FPR.

[0140] Figure 7E Table 5 is illustrated, which includes malicious URL classification test set with threshold configured to detect FPR. Decision threshold is configured to obey the maximum FPR.

[0141] ​Note that Table 3 can also be interpreted as investigating susceptibility to OOD attacks given that the model is trained without noise, and the task is then to classify a test set including noise. In Table 4, we see that IUPG maintains more performance than CCE over a one-year period, even with distribution shifts. In line with IUPG's central assumption, the noise-resistant nature of the IUPG network is naturally more robust to distribution shifts in benign classes. The prototype definition strategy appears to influence performance. Malicious URLs exhibit numerous and diverse clusters. Intuitively, we will see the benefits of defining a large number of prototypes using a base set.

[0142] As an additional study to our central hypothesis, we trained a single IUPG model (e.g., Figure 6 (B) and a larger stacked integration of the CCE network for JS classification resulted in a lower FNR on the integrated test set compared to the IUPG model at the same FPR. We accumulated data from top-ranking popular websites. A new set of 5M JS samples. Utilizing the thresholded test set FPR. We used VT to cross-reference all detections from these two models on this dataset (e.g., see Gaurav Sood's " virustotal: R Client for the virustotal API (2017, R package version 0.2.1)). High VTS indicates a strong consensus on maliciousness among a large number of industrial cybersecurity service providers. Table 5 shows the detection VTS for both models. Importantly, despite the opposing performance gaps in the test partitions, we see a significant shift towards higher VT consensus in IUPG detection. It is important to highlight this point given the prevalence of constructing TTV partitions from similar distributions but deploying the models in more complex environments.

[0143] Out-of-Distribution (OOD) Attack Simulation

[0144] In addition to the explorations in Table 3, we also explored the trends of IUPG and its CCE counterparts producing false positive (FP) responses to OOD inputs at decision thresholds representing in-distribution data confidence levels. Therefore, we are investigating different trends in model outputs on OOD samples compared to in-distribution samples with similar confidence levels. Our analysis shows... Figure 8 In, as further described below.

[0145] Figure 8 The figure illustrates the simulation results of an OOD attack. The Fibonacci retrieval rate (FPR) is measured on the OOD test set, where the decision threshold is based on all confidence scores generated on the target class test data. Percentile configuration. (A) An image classification model trained without noise on a Gaussian OOD test set. (B) An image classification model trained without noise on a random stroke OOD test set. (C) An image classification model trained with Gaussian noise on a random stroke OOD test set. (D) A binary JS classifier on a randomized benign JS OOD test set.

[0146] We found that the false positive rate (FPR) in the IUPG case is smaller with a large margin when a decision threshold representing a typical confidence level is applied to the target class test data. The lower propensity to produce FP on OOD content allows for the use of a more lenient decision threshold in real-world systems, resulting in higher recall. This result helps confirm that the classification performance gap widens under stricter FPR requirements, similar to what has been described above.

[0147] Additional attack simulation

[0148] We explored vulnerabilities in our JS malware classifier append attack. Our simulation results are shown in Table 6, as further described below. For each epoch, we also attempted to dynamically modify all non-benign classes so that 33% of all their members were appended with random benign fragments in the same TTV partition. The fragments were given a random size between 1000 and 5000 characters.

[0149] Figure 12A -C provides various examples of additional attacks that can evade detection by existing malware detection solutions. Figure 12A and 12B This illustration demonstrates why signature or hash matching is generally insufficient for effective malware detection, and why advanced ML and DL models should also be deployed to protect against "patient zero" malware (e.g., malicious scripts in these examples). Both are examples of the same malicious activity that is difficult to detect because it generates many unique scripts and uses different obfuscation techniques on the injected fragments. In fact, despite Figure 12B The example shown is a hash / SHA256 (i.e., (At the time of writing, it was already called VirtusTotal, but...) Figure 12A The example shown is a hash / SHA256 (i.e., The new element is one that has not been previously detected. A similar example can be shown for malware injection, where blank spaces and padding are added in an attempt to fool an existing ML classifier.

[0150] In addition to redirectors and droppers, the publicly disclosed IUPG framework is also effective and efficient in detecting JavaScript (JS) malware, such as phishing toolkits, clickjacking campaigns, malicious advertising libraries, and other exploit kits. For example, it has been found on over 60 websites. Figure 12C Similar scripts (e.g., a sample malware script with obfuscated phishing JavaScript in HTML is shown that generates a fake Facebook login page), such as Note that this script uses re-obfuscation techniques, but it can still accurately detect errors using a model trained with IUPG using publicly available techniques as described in this article.

[0151] Figure 9 Table 6 illustrates the results of the appended attack simulation. Each cell shows the percentage of malware for which the model produced a malicious decision on the original but a benign decision after appending a benign data fragment of a given size. Twenty random fragments were tested against each malware. The decision threshold was configured to conform to a maximum FPR of 0.1% on the test set. Adversarial training was conducted by… Marking.

[0152] We found a significant margin between the vulnerabilities of the IUPG and its CCE counterpart, with and without adversarial training. Importantly, note the binary... The model fails to protect against additional attacks that exceed the fragment size used during training. Note that the binary dataset contains hundreds of malware families with a single generic label, thus representing a significantly more difficult problem compared to multi-class classification. Our multi-class dataset has much less variability, and therefore, the extracted features are more specific—leading to less susceptibility to activations of noisy benign inputs. Additionally, note that black-box additional attacks can take the form of malicious injections of large benign files. On a real-world dataset of malicious JS injections, we find that the IUPG network increases the number of detections from 76 to 2259 on top of the aforementioned larger ensembles, as discussed in the classification performance section above. This practical result is confirmed by the results in Table 6.

[0153] Fast Gradient Signed Method (FGSM) Attack

[0154] To demonstrate the potential of combining IUPG with existing adversarial training techniques, we combine an image classifier with the Fast Gradient Sign Method (FGSM) (see, for example, Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy's "Explaining and Harnessing Adversarial Examples"). arXiv e-prints, page arXiv: 1412.6572, December 2014). We find that IUPG produces significantly greater resistance to FGSM attacks than its CCE counterpart, both with and without FGSM adversarial training. This is visualized in Figure 10A FIG. 4B.

[0155] Figure 10A-10B FIG. 4C illustrates accuracy versus FGSM perturbation scaling factor over test images that were correctly classified. Figure 10A FIG. 5 illustrates results using standard training. Figure 10B FIG. 6 illustrates results when training these models with the FGSM training procedure described above.

[0156] In particular, as shown in Figure 10B we use the following common FGSM training parameters: a , of 0.1 for MNIST and a , of 0.3 for Fashion MNIST. In line with our core hypothesis, IUPG networks by design should be less sensitive to low-level perturbations. This is especially due to the prototyping mechanism of IUPG, which encourages specialized sensitivity to high-level information shared between subsets of data. Thus, Figure 10B Both Table 6 and Table 7 demonstrate the superiority of using IUPG in combination with special adversarial training compared to using either in isolation. As such, to maximize success in various real-world environments, it is generally recommended to combine strengths, such as malware classification and similar applications of the disclosed IUPG technology.

[0157] Thus, we present the IUPG learning framework and demonstrate its impact on classification networks compared to CCE. Our core hypothesis is that the inherent noise-robustness and increased feature specificity of IUPG provide an elevated ability to properly handle OOD content. This feature logically connects all supporting results presented in this work: (1) increased or stable classification performance; (2) reduced performance loss due to recent bias; (3) decreased FP for OOD noise; and (4) reduced vulnerability to some noise-based attacks. Properly handling OOD content is generally important for models in benign real-world environments where limited samples cannot be reasonably leveraged, such as malware classification.

[0158] As noted above, the unique benefits of IUPG are particularly useful for malware classification efforts. In this context, additional attacks can lead to risky false negatives, while OOD failures can lead to high-cost false positives, which can be addressed or mitigated using the IUPG techniques described above. For example, as noted above, IUPG has been shown to increase resistance to several attack variants, as well as decrease FP response to OOD inputs. As such, IUPG can increase the efficiency and security of ML systems for such machine learning (ML) systems, such as malware classification and detection using a secure platform as described herein. Black-box attack defense is particularly relevant for attacks that exploit on proprietary systems, where an attacker can gain access to benign data but not model details. Security of deep learning and ML in general is often important to its adoption and effectiveness, especially in safety-critical environments. Particularly relevant are cybersecurity service providers, who can directly benefit from the ways in which IUPG is adopted, such as increased successful detection of malware, increased robustness to adversaries, increased customer trust, and advancing the common moral mission of securing the digital world.

[0159] Example JavaScript (JS) data collection for experiments

[0160] We collected benign JS with filters from popular websites. In particular, we used the top 1M domains from the Tranco list (e.g., publicly available at https: / / tranco-list.eu / ), which aggregates and cleans several popular lists, and as such, is considered by researchers as a more accurate and clean option (e.g., see V. L. Pochat, T. van Goethem, and W. Joosen, “Rigging research results by manipulating top websites rankings” (CoRR, abs / 1806.01156, 2018), available at http: / / arxiv.org / abs / 1806.01156). In addition to the filtering of Tranco, we also ignored five samples that were flagged by the most advanced commercial URL filtering services. We leveraged VirusTotal (VT) (e.g., see G. Sood, “virustotal: RClient for the virustotal API” (2017, URL https: / / www.virustotal.com. R package version 0.2.1) as the main source of malicious JS samples. Problematically, VT’s malicious file feed mostly contains HTML files, not JS scripts. To pinpoint malicious scripts inside HTML files accurately, we extracted inline code segments and externally referenced scripts from VT’s feed and submitted them again to be reconfirmed by VT. We required at least three VT vendor hits, which empirically showed to be quite accurate. Data collection was performed during 2014-2020. The most popular tokens among the labels are “ExpKit,” “Trojan,” “Virus,” “JS.Agent,” and “HTML / WebPhish.” To complement the malware data, we added malicious exploit kits provided to us by a large web and enterprise security company. To search for multi-class data on the problem of malware family labeling, we isolated nine subsets of the malicious data. These subsets were determined by clustering the malware data with the method further described in the section on the use of K-Means++ for IUPG. These clusters generally include malware families and some obfuscation techniques, whose output has a high visual similarity. Each malware cluster will now be described below.

[0161] 1. Angle exploit kit examples that aim to deliver malicious payloads through a web browser without any interaction from the victim (e.g., see I. Nikolaev, M. Grill, and V. Valeros, “Exploit kit website detection using http proxy logs,” in Proceedings of the Fifth International Conference on Network, Communication and Computing, ICNCC 2016, page 120-125, New York, NY, USA, 2016, Association for Computing Machinery, ISBN 9781450347938, doi: 10.1145 / 3033288.3033354, available at https: / / doi.org / 10.1145 / 132 3033288.3033354; B. Duncan, “Understanding angler exploit kit - part 1: Exploit kit fundamentals,” June 2016, URL https: / / unit42.paloaltonetworks.com / unit42-understanding-angler exploit-kit-part-1-exploit-kit-fundamentals / ; F. Howard, “A closer look at the angler-exploit-kit,” July 2019, available at https: / / news.sophos.com / en-us / 2015 / 07 / 21 / a-closer-look-at-the-angler-exploit-kit / ; and A. Zaharia, “The ultimate guide to angler exploit kit for non-technical people [updated],” February 2017, URL https: / / heimdalsecurity.com / blog / ultimate-guide-angler-exploit-kit-non-technical-people / . Has both high token-level and character-level randomness.

[0162] 2. “Hea2p” style obfuscation, which is often associated with phishing kits (see, e.g., O. Starov, Y. Zhou, and J. Wang, “Detecting malicious campaigns in obfuscated javascript with scalable behavioral analysis,” pages 218-223, May 2019, doi: 10.1109 / SPW.2019.00048). Has high token-level similarity but a lot of character-level randomness.

[0163] 3. Clickjackers, which focus on manufacturing artificial “like” button presses on social media websites (see id. ). Has high token-level similarity but a lot of character-level randomness. Has both high token-level and character-level randomness.

[0164] 4. “Lololo” style obfuscation, which produces output with low token structure similarity but contains recognizable character-level patterns.

[0165] 5. Nemucod, which is a threat family that attempts to download and install other malware onto a device, including ransomware (see, e.g., ). Has both high token-level and character-level randomness.

[0166] 6. Multiple unnamed JS packers, which produce output with both high token-level and character-level randomness, given that the packed code can exist anywhere in the original script.

[0167] 7. Multiple unnamed JS trojans (see, e.g., C. E. Landwehr, A. R. Bull, J. P. McDermott, and W. S. Choi, “A taxonomy of computer program security flaws,” ACM Comput. Surv., 26(3):211-254, September 1994, ISSN 0360-0300. doi: 10.1145 / 185403.185412, URL https: / / doi.org / 10.1145 / 185403.185412), which have high token-level similarity but a lot of character-level randomness.

[0168] 8. Another multiple unnamed JS trojans, which have high token-level similarity but a lot of character-level randomness (see id. ).

[0169] 9. Several unnamed cryptographic techniques that produce outputs with high token-level similarity but a lot of character-level randomness.

[0170] Figure 11 This is an example t-SNE visualization based on some embodiments of the U-vector space. Specifically, Figure 11 The visualization shown is a real-world example of the output vector space after training a multi-class JS malware family classifier. The network was trained to identify nine different JS malware families listed in the illustration, with benign classes that deviate from the target. Each of the nine target malware family classes is tightly grouped around a single specified prototype, while the benign data is mapped more randomly to the center. This visualization was generated using t-SNE on the mapping representation of the prototypes in the validation data and output vector space (see, for example, van der Maaten & Hinton, GE (2008) “Visualizing High-Dimensional Data Using t-SNE” Journal of Machine Learning Research, 9 (November), pages 2579-2605).

[0171] Use of IUPG

[0172] For our IUPG experiments, we utilized K-means++ (e.g., see D. Arthur and S. Vassilvitskii, “K-means++: The advantages of careful seeding”, in Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA2007, pages 1027–1035, USA, 2007, Society for Industrial and Applied Mathematics, ISBN 9780898716245). Recall that clustering was used to discover intelligent IUPG prototype initialization. We used K-means++ (see...). id.) for both the intra-work clustering of MNIST (e.g., see Y. LeCun and C. Cortes, “MNIST database of handwritten digits” (2010, URL http: / / yann.lecun.com / exdb / mnist / ) and the malicious JS dataset.

[0173] MNIST clustering

[0174] We first group each digit class within the training data together and cluster each subgroup separately. On each digit subgroup, we project each image onto the first 75 principal component vectors using Principal Component Analysis (PCA) (e.g., see I. Jolliffe and Springer- Verlag, “Principal Component Analysis” (Springer Series in Statistics, Springer, 2002, ISBN 9780387954424, ) and perform K-means++ clustering across these compressed representations with K = 1. We use Euclidean distance for clustering. We then compute the Euclidean distance of the resulting cluster centers to all images within the digit subgroup. The training image closest to the cluster center is selected as the prototype initialization. The selected image pixels are slightly perturbed with Gaussian noise to avoid potential overfitting.

[0175] Malicious JS clustering

[0176] Since all benign data is assigned an off-target label, we first group the 54 all-malicious samples together. For the multi-class model, we group the malware families together and cluster each family separately. For the binary model, we cluster all malware samples simultaneously. We further isolate only the token sequence representation of each malware sample We vectorize each token index sequence by computing term frequency inverse document frequency (TF-IDF) (see, e.g., C. Sammut and G. I. Webb, editors, TF-IDF, pages 986-987, Springer US, Boston, MA, 2010, ISBN 978-0-387-30164-8, doi:10.1007 / 978-0-387-30164-8_832, available at https: / / doi.org / 10.1007 / 978-0-387-30164-8_832) vectors over the token vocabulary. We perform K-means++ on these TF-IDF vector representations for each malicious sample, where for binary models and for multiclass models We use Euclidean distance for clustering. We then compute the Euclidean distance of the resulting cluster center(s) to all malware samples in the set. The training malware sample closest to the cluster center is chosen as the prototype initialization. After initializing all embedded vectors, each prototype is initialized to the corresponding from the selected malware sample. Each is slightly perturbed with Gaussian noise to avoid potential overfitting.

[0177] Example IUPG framework for malware JavaScript classification

[0178] Figure 13An IPUG framework for malware JavaScript classification is illustrated in accordance with some embodiments. JavaScript documents 1302 are tokenized using an OpenNMT Tokenizer 1304 (e.g., an open source tokenizer available at https: / / github.com / OpenNMT / Tokenizer) to generate characters (character tokens) 1306 encoded as character encoding 1310 and tokens 1308 encoded as token encoding 1312. Tokenization is followed by integration of CNN feature extractors, including a CCE CNN feature extractor 1314 and an IUPG CNN feature extractor 1316 (e.g., implemented similarly using the disclosed IUPG techniques that can be applied in the context of JS malware classification, as described above), followed by an XGB classifier 1318 to generate JS malware classification rulings as shown at 1320 based on the integration of CNN feature extractors using a combination of the CCE CNN and IUPG CNN classification techniques described above, to facilitate more effective and efficient JS malware detection solutions (e.g., more robust against potential adversarial evasion techniques, such as additional attacks similarly as described above).

[0179] The disclosed IUPG framework and techniques can similarly be applied to URL classification, such as for URL filtering security solutions, and / or other computer / network security classification / detection for various security solutions, as will be apparent to those of ordinary skill in the art.

[0180] Example process embodiments for performing the disclosed IUPG techniques will now be further described below.

[0181] Example process embodiments for malware classification using an innocent until proven guilty model

[0182] Figure 14is an example of a process that performs static analysis of a sample for malware classification using an innocent until proven guilty (IUPG) model in accordance with some embodiments. In some embodiments, the process 1400 is performed by the security platform 122, and in particular by the analyzer and detector 154. For example, the analyzer and detector 154 can be implemented using a script (or set of scripts) written in an appropriate scripting language (e.g., Python). In some embodiments, the process 1400 is performed by the data appliance 102, and in particular by the threat engine 244. For example, the threat engine 244 can be implemented using a script (or set of scripts) written in an appropriate scripting language (e.g., Python). In some embodiments, the process 1400 can also be performed on an endpoint such as the client device 110 (e.g., by an endpoint protection application executing on the client device 110). In some embodiments, the process 1400 can also be performed by a cloud-based security service, such as using the security platform 122, as further described below.

[0183] The process 1400 begins at 1402 when a set of one or more IUPG models for security analysis are stored on the network device. For example, the IUPG models, such as IUPG models for JS code, HTML code, and / or other programming / scripting languages, as well as other structured text such as URLs or unstructured content such as images, can be generated (e.g., and / or periodically updated / replaced) based on training and validation data using the techniques described above.

[0184] At 1404, a static analysis of content associated with a sample received at the network device is performed using at least one stored IUPG classification model. As one example of the processing performed at 1404, such as for the data appliance 102 and / or the client device 110, for a given session, when a protocol decoder detects the start of a file, the associated protocol decoder can invoke or otherwise utilize an appropriate file-specific decoder. As explained above, the file type is determined (e.g., by the decoder 402) and associated with the session. In another example implementation, the file can be sent to a cloud-based security service (e.g., a commercially available cloud-based security service such as the WildFire® malware analysis environment by Palo Alto Networks, Inc., which is a commercially available cloud security service provided by Palo Alto Networks, Inc. that includes automated security analysis of malware samples as well as security expert analysis, or a similar solution provided by another vendor) for further analysis. TM cloud-based security service such as the WildFire® malware analysis environment by Palo Alto Networks, Inc., which is a commercially available cloud security service provided by Palo Alto Networks, Inc. that includes automated security analysis of malware samples as well as security expert analysis, or a similar solution provided by another vendor).

[0185] At 1406, it is determined whether the sample is malicious based at least in part on the static analysis of the content associated with the received sample. In example implementations, an appropriate IUPG model (e.g., an IUPG model applied to JS code for JS samples, an IUPG model applied to HTML code for HTML samples, etc.) is used to determine whether the file's class verdict is malicious or benign (i.e., compare the final value obtained using the IUPG model in combination with one or other classification models, such as a CCE CNN model trained for the appropriate content, such as similarly described above).

[0186] At 1408, a security policy based action is performed in response to determining that the sample is malicious. In particular, responsive to the determination made at 1406, an action is taken. One example of a responsive action is terminating the session, such as for the data appliance 102 and / or the client device 110. Another example of a responsive action is allowing the session to continue, but preventing the file from being accessed and / or transmitted (and instead being placed in a quarantine area), such as for the data appliance 102 and / or the client device 110. Yet another example of a responsive action is sending a determination that the sample is malicious to a subscriber (e.g., the data appliance 102 and / or the client device 110) that submitted the sample for analysis, such as for the security platform 122, to notify the subscriber that the sample was determined to be malicious so that the subscriber can perform a response based on a locally configured security policy. In various embodiments, the security platform 122, appliance 102, and / or client device 110 are configured to share their verdicts (whether benign verdicts, malicious verdicts, or both) with one or more other devices / platforms (e.g., the security platform 122, appliance 102, and / or client device 110, etc.). As an example, when the security platform 122 completes its independent analysis of a sample, it can use the verdict reported by the appliance 102 for a variety of purposes, including evaluating the performance of the models that formed the verdict.

[0187] In example embodiments, the security platform 122 is configured to target a particular false positive rate (e.g., 0.01%) when generating models for use by appliances such as the data appliance 102. Thus, in some cases (e.g., one in every thousand files), the data appliance 102 can falsely determine that a benign file is malicious when performing inline analysis using a model according to the techniques described herein. In such scenarios, if the security platform 122 subsequently determines that the file is actually benign, it can be added to a whitelist so that it is not subsequently flagged as malicious (e.g., by another appliance).

[0188] Example process embodiments for building adversarial and false positive resistant deep learning models for security solutions

[0189] Figure 15 is an example of a process for generating an innocent until proven guilty (IUPG) model for malware classification according to some embodiments. In particular, Figure 15 An example process for generating an innocent until proven guilty (IUPG) model for malware classification is depicted in FIG. 15. In various embodiments, the process 1500 is performed by the security platform 122 (e.g., using the model builder 152).

[0190] The process 1500 begins at 1502 when training data is received (e.g., the training data includes a set of files for proper training context, such as JS files, HTML files, URLs, etc.) that trains an innocent until proven guilty (IUPG) model for classifying malicious content and benign content based on static analysis.

[0191] At 1504, a set of tokens is extracted from the set of input files to generate character encodings and token encodings. As described above, various techniques are disclosed for tokenizing content based on a set of characters and other tokens extracted from a set of input files, such as JS files.

[0192] At 1506, an IUPG CNN feature extractor is generated. As similarly described above, additional feature vectors based on different levels of abstraction / layers can also be generated based on different representations to be extracted from the set of input files.

[0193] At 1508, integration of the IUPG CNN feature extractor with one or more other CNN-based feature extractors (e.g., a CCE CNN feature extractor or another form of CNN-based feature extractor) is performed for classifying malicious content and benign content based on static analysis of samples. In one embodiment, after integration of the IUPG and CCE-based CNN feature extractors, an XGB classifier is generated, such as for classifying malicious JS content and benign JS content based on static analysis of samples, as similarly described.

[0194] As also similarly described above, various IUPG models for one or more programming / scripting languages or other content can be built using open source or other tools, and where applicable, hyperparameter tuning as described above is performed, which may, for example, be tuned to efficiently perform these IUPG models for static analysis-based classification of samples to be performed / enacted on various computing environments that can have different computing resources (e.g., memory resources, processor / CPU resources, etc. available to process these IUPG models). Moreover, IUPG models (e.g., generated by the model builder 152 using the process 1500) can be sent (e.g., as part of a subscription service) to the data appliance 102, the client devices 110, and / or other applicable recipients (e.g., the data appliances 136 and 148, etc.).

[0195] In various embodiments, the model builder 152 generates IUPG models on a daily or other applicable / periodic basis (e.g., IUPG models for one or more types of source code, i.e., different programming / scripting languages such as JS, HTML, etc., and / or other as described above). By performing the process 1500 or otherwise generating models on a periodic basis, the security platform 122 and / or cloud-based security service can help ensure that the various security classification models detect the most recent types of malware threats (e.g., those recently deployed by nefarious individuals).

[0196] While the foregoing embodiments have been described in some detail for purposes of clarity and the like, it will be apparent that certain modifications can be made without departing from the scope of the application. Accordingly, the disclosed embodiments are to be considered as illustrative and not restrictive.

Claims

1. A system for identifying malware based on an innocent until proven guilty (IUPG) model, comprising: a processor configured to: store, on a networked device, a set of one or more innocent until proven guilty (IUPG) models comprising static analysis for a sample, wherein the IUPG model is an adversarial and affirmative deep learning model; receive training data that trains an IUPG model for classifying malicious content and benign content based on static analysis; extract a set of tokens from a set of input files to generate character encodings and token encodings; generate an IUPG CNN feature extractor; perform integration of the IUPG CNN feature extractor with one or more other CNN-based feature extractors for classifying malicious content and benign content based on static analysis of a sample; perform static analysis of content associated with a sample, wherein performing static analysis of content comprises using at least one stored IUPG model and another type of CNN-based classifier, wherein the at least one stored IUPG model is selected based at least in part on a file type associated with the sample, wherein performing static analysis of content associated with the sample comprises: combining the at least one stored IUPG model with the another type of CNN-based classifier to obtain a classifier; performing the static analysis using the classifier; and determining that the sample is malicious based at least in part on the static analysis of content associated with the sample, and performing an action based on a security policy in response to determining that the sample is malicious; and a memory coupled to the processor and configured to provide instructions to the processor.

2. The system of claim 1, wherein, the processor is configured to enumerate source code associated with a sample.

3. The system of claim 1, wherein, the processor is configured to enumerate source code associated with a sample into a set of characters.

4. The system of claim 1, wherein, the processor is configured to enumerate source code associated with a sample into a set of characters and a set of tokens.

5. The system of claim 1, wherein, the processor is configured to enumerate source code associated with a sample into a set of characters, a set of tokens, and an abstract syntax tree (AST).

6. The system of claim 1, wherein, the processor is further configured to determine a file type associated with the sample.

7. The system of claim 1, wherein, the processor is configured to determine a file type associated with the sample and select from a set of one or more IUPG models based on the determined file type associated with the file.

8. The system of claim 1, wherein, the processor is further configured to receive at least one updated classification model.

9. The system of claim 1, wherein, the processor is further configured to receive another IUPG model for another programming language.

10. A method for identifying malware based on an innocent until proven guilty (IUPG) model, comprising: storing a set of one or more innocent until proven guilty (IUPG) models comprising static analysis for a sample, wherein the IUPG model is an adversarial and affirmative deep learning model; receiving training data that trains an IUPG model for classifying malicious content and benign content based on static analysis; extracting a set of tokens from a set of input files to generate character encodings and token encodings; generating an IUPG CNN feature extractor; performing integration of the IUPG CNN feature extractor with one or more other CNN-based feature extractors for classifying malicious content and benign content based on static analysis of a sample; performing an integration of the IUPG CNN feature extractor with one or more other CNN-based feature extractors for classifying malicious content and benign content based on static analysis of a sample; performing static analysis of content associated with a sample, wherein performing the static analysis of the content includes using at least one stored IUPG model and another type of CNN-based classifier, wherein the at least one stored IUPG model is selected based at least in part on a file type associated with the sample, wherein the performing the static analysis of the content associated with the sample includes: combining the at least one stored IUPG model with the other type of CNN-based classifier to obtain a classifier; and performing the static analysis using the classifier; and determining that a sample is malicious based at least in part on the static analysis of the content associated with the sample, and performing an action based on a security policy in response to determining that the sample is malicious.

11. The method of claim 10, further comprising: enumerating source code associated with a sample.

12. The method of claim 10, further comprising: enumerating source code associated with a sample into a character set.

13. The method of claim 10, further comprising: enumerating source code associated with a sample into a character set and a token set.

14. The method of claim 10, further comprising: determining a file type associated with a sample.

15. The method of claim 10, further comprising: determining a file type associated with a sample; and selecting from a set of one or more IUPG models based on the determined file type associated with the file.

16. The method of claim 10, further comprising: receiving at least one updated IUPG model.

17. The method of claim 10, further comprising: receiving another IUPG model for another programming language.

18. A computer program product embodied in a tangible computer-readable storage medium and comprising computer instructions for: storing a set of one or more innocent-until-proven-guilty IUPG models for static analysis of a sample, wherein the IUPG models are deep learning models that are adversarial and affirmative; receiving training data that trains an IUPG model for classifying malicious content and benign content based on static analysis; extracting a token set from an input file set to generate character encodings and token encodings; generating an IUPG CNN feature extractor; performing an integration of the IUPG CNN feature extractor with one or more other CNN-based feature extractors for classifying malicious content and benign content based on static analysis of a sample; performing static analysis of content associated with the sample, wherein performing static analysis of content comprises using at least one stored IUPG model and another type of CNN-based classifier, wherein the at least one stored IUPG model is selected based at least in part on a file type associated with the sample, wherein performing static analysis of content associated with the sample comprises: combining the at least one stored IUPG model with the another type of CNN-based classifier to obtain a classifier; performing the static analysis using the classifier; and determining that the sample is malicious based at least in part on the static analysis of content associated with the sample, and performing an action based on a security policy in response to determining that the sample is malicious.

Images