Anti-malware system and method using optimal triggering of artificial intelligence modules

Abstract

Some embodiments determine optimal trigger events for executing malware-detection artificial intelligence (AI) modules by tracking the performance of each module over the lifetime of each software sample in the training corpus. The computational cost associated with each AI module can be significantly reduced by only running each module sporadically in response to detecting the occurrence of one of the identified trigger events. In some embodiments, the malware detector employs multiple AI modules, each with its own unique trigger. The trigger selection can be updated at any time without affecting other aspects of the malware detector.

Description

[Technical Field]

[0001]

[0001] The present invention relates to computer security, and in particular to using artificial intelligence in detecting malicious software. [Background technology]

[0002] Malicious software, also known as malware, affects many computer systems worldwide. In its many forms, such as computer viruses, worms, rootkits, unwanted adware, ransomware, and spyware, malware poses serious risks to millions of computer users, making them vulnerable to, among other things, extortion, loss of data and confidential information, identity theft, and lost productivity. Malware may also display material that some users consider immoral, excessively violent, harassing, or otherwise objectionable. The explosive growth of mobile computing has only exacerbated the exposure and associated risks, with millions of devices, such as smartphones and tablet computers, constantly connected to the Internet and acting as potential targets for malware.

[0003]

[0003] Security software can be used to detect malware that infects a user's computer system and, additionally, to remove or prevent the execution of such malware. Several malware detection techniques are known in the art. Modern methods leverage recent developments in artificial intelligence (AI) and machine learning (ML) and rely on artificial neural networks pre-trained on a corpus of known malicious and non-infected samples. In a typical example, the neural network receives a vector of feature values ​​characterizing target software entities and outputs a label indicating whether each target software entity is malicious.

[0004] Despite these advantages, such AI systems face significant technical challenges. One example is the selection of input features. Typically, there is no clear or universal metric for selecting which features of target software are more likely to reveal malicious intent and / or distinguish malicious from non-malicious behavior. Another challenge facing AI-enabled anti-malware solutions is the availability of input features. In other words, even when a set of relevant features is known, some of the respective features may not be available at all times and / or on all client devices. This is particularly true for mobile computing platforms, such as smartphones and wearables, where extracting or evaluating certain features may require specific permissions from the user.

[0005]

[0005] To address feature selection and availability issues, some conventional approaches increase feature count and therefore AI model size in an attempt to improve AI model performance. Yet, large neural networks are notoriously expensive to train and typically require large training corpora that are difficult to acquire, annotate, and maintain. Furthermore, implementing and running large AI models on some client systems, such as mobile computing devices, can generate significant computational overhead, negatively impacting the user experience.

[0006] For all of the reasons outlined above, there is considerable effort and sustained interest in developing robust and scalable computer security systems and methods capable of responding quickly and efficiently to emerging threats. Summary of the Invention

[0007] According to one aspect, a computer system includes at least one hardware processor configured to determine a trigger event for executing a malware-detection artificial intelligence (AI) module. Determining the trigger event includes determining a plurality of provisional verdicts indicating whether a training software entity is malicious, where each of the plurality of provisional verdicts is associated with a distinct event of a plurality of events caused by execution of the training software entity. Determining each verdict includes updating a feature vector characterizing the training software entity according to the distinct event and executing the AI ​​module to determine the respective verdict according to the updated feature vector. Determining the trigger event further includes selecting a trigger event from the plurality of events according to the plurality of provisional verdicts in response to determining the plurality of provisional verdicts. The at least one hardware processor is further configured to configure the malware detector to selectively execute another instance of the AI ​​module in response to occurrence of the trigger event to determine whether the target software entity is malicious.

[0008] According to another aspect, a computer security method includes determining, using at least one hardware processor of a computer system, a trigger event for executing a malware detection AI module. The determining the trigger event includes determining a plurality of provisional verdicts indicating whether a training software entity is malicious, where each of the plurality of provisional verdicts is associated with a distinct event of a plurality of events caused by execution of the training software entity. The determining each verdict includes updating a feature vector characterizing the training software entity according to the distinct event and executing the AI ​​module to determine the respective verdict according to the updated feature vector. The determining the trigger event further includes, in response to determining the plurality of provisional verdicts, selecting a trigger event from the plurality of events according to the plurality of provisional verdicts. The method further includes, in response to determining the trigger event, configuring the malware detector, using the at least one hardware processor, to selectively execute another instance of the AI ​​module in response to occurrence of the trigger event to determine whether the target software entity is malicious.

[0009] According to another aspect, a non-transitory computer-readable medium stores instructions that, when executed by at least one hardware processor of the computer system, cause the computer system to determine a trigger event for executing a malware-detection AI module. Determining the trigger event includes determining a plurality of provisional verdicts indicating whether a training software entity is malicious, where each of the plurality of provisional verdicts is associated with a distinct event of a plurality of events caused by execution of the training software entity. Determining each verdict includes updating a feature vector characterizing the training software entity according to the distinct event and executing the AI ​​module to determine the respective verdict according to the updated feature vector. Determining the trigger event further includes, in response to determining the plurality of provisional verdicts, selecting a trigger event from the plurality of events according to the plurality of provisional verdicts. The instructions further cause the computer system, in response to determining the trigger event, to configure the malware detector to selectively execute another instance of the AI ​​module in response to occurrence of the trigger event to determine whether the target software entity is malicious.

[0010] The foregoing aspects and advantages of the present invention will be better understood upon reading the following detailed description and upon reference to the drawings in which: [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram of multiple client devices protected from malware according to some embodiments of the present invention. [Figure 2]

[0012] FIG. 2 is a diagram of exemplary software executing on a client device in accordance with some embodiments of the present invention. [Figure 3]

[0013] FIG. 2 is a diagram of exemplary components of a computer security module according to some embodiments of the present invention. [Figure 4]

[0014] FIG. 1 is a diagram of an exemplary artificial intelligence (AI) module that receives input feature vectors and outputs a preliminary security verdict in accordance with some embodiments of the present invention. [Figure 5]

[0015] FIG. 2 is a diagram of an exemplary set of feature vectors and associated feature extractors according to some embodiments of the present invention; [Figure 6]

[0016] FIG. 4 illustrates an exemplary sequence of steps performed by a module activation engine according to some embodiments of the present invention. [Figure 7]

[0017] FIG. 10 illustrates another exemplary sequence of steps performed by a module activation engine according to some embodiments of the present invention. [Figure 8]

[0018] FIG. 4 illustrates an exemplary sequence of steps performed by a set of feature extractors according to some embodiments of the present invention. [Figure 9]

[0019] FIG. 10 illustrates an exemplary sequence of steps performed by a trigger engine according to some embodiments of the present invention. [Figure 10]

[0020] FIG. 3 illustrates an exemplary sequence of steps performed by a malware detector according to some embodiments of the present invention. [Figure 11]

[0021] FIG. 1 is a diagram of an exemplary AI training system in accordance with some embodiments of the present invention. [Figure 12]

[0022] FIG. 10 illustrates an exemplary procedure for training an AI module according to some embodiments of the present invention. [Figure 13]

[0023] FIG. 1 illustrates an exemplary sequence of steps performed by an AI training system to identify optimal trigger events according to some embodiments of the present invention. [Figure 14]

[0024] FIG. 1 is a diagram of an exemplary hardware configuration for a computer system programmed to perform some of the methods described herein. DETAILED DESCRIPTION OF THE INVENTION

[0012]

[0025] In the following description, it should be understood that all recited connections between structures can be direct operational connections or indirect operational connections through intermediate structures. A set of elements includes one or more elements. Any enumeration of an element is understood to refer to at least one element. A plurality of elements includes at least two elements. Any use of "or" means a non-exclusive or. Unless otherwise required, any described method steps need not necessarily be performed in the particular illustrated order. A first element (e.g., data) derived from a second element encompasses a first element equal to the second element, as well as a first element generated by processing the second element and, optionally, other data. Making a decision or determination according to a parameter encompasses making a decision or determination according to the parameter and, optionally, other data. Unless otherwise specified, an indicator of some quantity / data may be the quantity / data itself or an indicator different from the quantity / data itself. A computer program is a sequence of processor instructions that perform tasks. Computer programs described in some embodiments of the present invention may be standalone software entities or subentities (e.g., subroutines, libraries) of other computer programs. Unless otherwise specified, a process is an instance of a computer program and is characterized by having at least a thread of execution and a virtual memory space allocated to the thread of execution, the contents of each virtual memory space containing executable code. The term "database" is used herein to refer to any organized, searchable collection of data. The term "predicate" refers to a statement that has varying degrees of truth depending on the values ​​of the predicate variables. Evaluating a predicate includes determining the truth value of each predicate. Computer-readable media encompass non-transitory media such as magnetic, optical, and semiconductor storage media (e.g., hard drives, optical disks, flash memory, DRAM), as well as communication links such as conductive cables and fiber optic links.According to some embodiments, the present invention provides, among other things, computer systems comprising hardware (e.g., one or more processors) programmed to perform the methods described herein, and computer-readable media encoded with instructions for performing the methods described herein.

[0013]

[0026] The following description illustrates embodiments of the present invention by way of example, and not necessarily by way of limitation.

[0027] 1 illustrates multiple client devices 12a-d protected from malware according to some embodiments of the present invention. Exemplary client devices 12a-d include personal computer systems, corporate mainframe computers, mobile computing platforms (e.g., laptop computers, tablets, smartphones), entertainment devices (e.g., TVs, game consoles), wearable devices (e.g., smartwatches, fitness bands), household appliances (e.g., thermostats, refrigerators), and any other electronic devices equipped with a processor, memory, and a communications interface that enables the respective device to communicate with other devices / computer systems. In some embodiments, each client device 12a-d includes a security module configured to detect malicious software. The security module may be embodied as a software application executing on the respective device's processor.

[0014]

[0028] Client devices 12a-d are connected to a communications network 15, which may include a local area network (e.g., a home network, an enterprise network, etc.), a wide area network, and / or the Internet. Network 15 collectively represents a set of hardware (physical layer) and software interfaces that enable the transport of data between devices 12a-d and other entities connected to network 15.

[0015]

[0029] 1 further shows a security server 14 connected to a communications network 15. The server 14 generally represents a set of communicatively coupled computer systems that may or may not be physically near one another. In some embodiments, as described below, a security module executing on each client device 12a-d may collaborate with the server 14 to protect each respective device. Stated differently, computer security activities may be divided between components of each device and the server 14. The server 14 may protect multiple client devices 12a-d, for example, pursuant to a service agreement / subscription, etc.

[0016]

[0030] Figure 2 shows exemplary software executing on a client device 12, which generally represents any of client devices 12a-d of Figure 1. An operating system (OS) 22 provides an interface between the hardware of client device 12 and other computer programs executing on the respective device. Exemplary operating systems include Windows®, Linux®, iOS®, and Android®, among others.

[0017]

[0031] In some embodiments, computer security module 30 protects client device 12 from computer security threats, such as malicious software and intrusions. The following description focuses on an exemplary embodiment in which module 30 comprises a set of computer programs, i.e., software executing on a processor of client device 12. Nevertheless, those skilled in the art will recognize that the description can be adapted to alternative embodiments in which module 30 is implemented in hardware or a combination of hardware and software without affecting the scope of the invention. Module 30 may form part of a larger security suite that also provides traffic control (e.g., firewall, parental control) and spam control services, among others.

[0018]

[0032] In some embodiments, computer security module 30 is configured to monitor the behavior of target software entities 24 to determine whether the entities 24 are malicious. In some embodiments, monitoring the behavior of the software includes detecting a set of events caused by the execution of each piece of software and analyzing each event as described in detail below. A single instance of security module 30 may be configured to monitor many (e.g., hundreds) of target entities 24 simultaneously.

[0019]

[0033] The target software entity 24 collectively represents any computer program, such as word processing, image processing, spreadsheet, calendar, gaming, social media, web browser, and electronic communication applications, among others. In some embodiments, the entity 24 comprises an individual computer process. Alternatively, the entity 24 may comprise a group of processes related by derivation and / or code injection relationships, as described, for example, in U.S. Pat. No. 10,706,151 (B2) to G. Hajmasan et al., entitled "Systems and Methods for Tracking Malicious Behavior Across Multiple Software Entities." In such embodiments, an event caused by any member of the group of related processes may be considered to represent the behavior of the group. The composition of the entity 24 may change over time as some constituent processes are terminated and / or others are added following spawning and / or injection. In some embodiments, a subject software entity 24 may be monitored even after some of its constituent processes have terminated, as long as the entity 24 still comprises at least one live process.

[0020]

[0034] 3 illustrates exemplary components of security module 30 according to some embodiments of the present invention. Module 30 includes a malware detector 44 that uses artificial intelligence (AI) techniques to determine whether target entity 24 is malicious. In an exemplary embodiment, malware detector 44 may implement multiple AI modules 50 and selectively execute modules 50 when triggered by the occurrence of specific events caused by the execution of target entity 24.

[0021]

[0035] In some embodiments, such as illustrated in FIG. 4 , each AI module 50 is configured to input a feature vector 52 and output a preliminary verdict 54, determined according to the feature vector 52, indicating whether the target entity 24 is malicious. Each target entity 24 is characterized by its own feature vector 52, which includes multiple feature values ​​53 comprising values ​​of various features / attributes of the respective target entity. Such features herein broadly refer to any attributes known in the art of computer security that can be used to determine whether a client device 12 contains malicious software. Some features may be static, i.e., the value of a feature may not change during the lifetime of the respective target entity. Examples of static features include file names, entity types (e.g., Windows executable files vs. Microsoft Word files), contents of sections of memory, contents of files or folders (e.g., Android manifest files), OS settings, text displayed to a user, various characteristics of images displayed by the target application 24 on the screen of the client device 12, etc. Other characteristics may be behavioral / dynamic in the sense that they characterize a set of actions of the respective entity, such as opening a file, changing access permissions, launching a child process, injecting code into another software entity, sending electronic communications (e.g., an HTTP request for a remote resource), etc. Some characteristic characteristics may be indicative of malware on their own, while others may indicate malicious intent only when evaluated together with other characteristics. Still other characteristic characteristics may indicate that the respective entity is non-malicious. One such example includes an entity being digitally signed by its creator; such a signature often serves as a mark of authenticity and non-infection.

[0022]

[0036] The components of feature vector 52 (i.e., the set of characteristic features) may be unique to each AI module 50. In other words, each AI module 50 may determine whether an entity 24 is malicious according to a distinct “perspective,” i.e., a distinct set of features and measures. Nevertheless, some features may be shared among multiple AI modules, as further described below. While the illustrated feature vector 52 in FIG. 4 comprises only Boolean values, those skilled in the art will recognize that this aspect is not meant to be limiting. Other exemplary feature values ​​53 may include numbers, strings of characters, and the like. Similarly, preliminary verdict 54 may comprise a Boolean value (e.g., malicious / clean) or a number indicating the likelihood / probability that the respective entity 24 is malicious. In yet other exemplary embodiments, verdict 54 may comprise multiple values, each indicating the likelihood that entity 24 belongs to a respective class / category of malicious behavior (e.g., ransomware, exfiltrator, etc.).

[0023]

[0037] The AI ​​module 50 may be implemented using any method known in the art of artificial intelligence and machine learning. For example, the AI ​​module may comprise a set of pre-trained artificial neural networks (NNs). Several NN architectures and strategies for training NNs, such as malware detectors, are known in the art. Simple examples include feedforward networks, perceptrons, support vector machines, and decision forests. Another exemplary NN, known as an autoencoder, can be trained to perform lossy compression and subsequent decompression of a set of feature values. Some embodiments may train an autoencoder with data obtained during the execution of benign software and use mismatches between the autoencoder's input and output as indicators of anomalous and therefore potentially malicious software behavior. Some AI modules 50 may use neural networks specialized in text and natural language processing to analyze features extracted from software configuration files, event logs, Android® manifest files, and / or user interfaces. Such modules may implement recurrent neural network (RNN) and long short-term memory (LSTM) architectures, among others. Other exemplary AI modules 50 may use specialized image processing NNs (such as convolutional neural networks, among others) to analyze screen snapshots and / or image files received or sent by respective client devices, the specific structural and functional details of such NNs being beyond the scope of this description.

[0024]

[0038] In some embodiments, feature values ​​53 are provided by a plurality of feature extractors 42, which may be embodied as computer programs executing on a processor of client device 12. Evaluating static features may include, for example, feature extractors 42 accessing OS-specific data structures such as the Windows® registry to read current values ​​of various settings, scanning various files and / or memory regions for unique contents (e.g., malware signature matching via hashing or other methods), etc. Evaluating dynamic / behavioral features may include determining whether target entity 24 has performed a unique action and updating feature values ​​53 accordingly. Execution of extractors 42 may be selectively triggered by the occurrence of various events, as described in more detail below.

[0025]

[0039] In some embodiments, hardware and software events are detected by an event processing infrastructure comprising a set of event detectors 32, a set of event processors 34, and an event dispatcher connected to the event detectors 32 and the event processors 34. The event processing infrastructure may comprise any implementation of a message delivery system. For example, the components 32-34-36 may register callbacks to be notified whenever a unique event occurs on the client device 12, and may further associate each event with the software entity that caused the respective event.

[0026]

[0040] Event detectors 32 comprise hardware and / or software devices configured to detect various events that occur during the execution of software on client device 12. Some detectors 32 may specialize in detecting particular types or categories of events. Exemplary detected events include application installation, uninstallation, and updates, process / application launch and termination, spawning of child processes (e.g., forking), dynamic loading / unloading of libraries, execution of specific processor instructions (e.g., system calls), file events such as file creation, writing, and deletion, and setting various OS parameters (e.g., Windows® registry events, permission / privilege changes), among others. Other exemplary detected events may include, among others, receiving a request to access a peripheral device (e.g., hard disk, SD card, network adapter, microphone, camera), receiving an incoming communication (e.g., Short Message Service - SMS message), a request to access a remote resource (e.g., a HyperText Transfer Protocol - HTTP request to access a particular URL, an attempt to access a document repository over a local network), a request framed with a particular uniform resource identifier scheme (e.g., a mailto: or ftp: request), and an attempt to send an electronic message (e.g., email, SMS, etc.). Still other exemplary events include moving the user interface / window of the target application 24 into and / or out of focus / foreground.

[0027]

[0041] Some embodiments of event detector 32 may further detect various timing-related events, such as periods of inactivity, i.e., time gaps between events, and / or time intervals during which the respective client device is not running, registering no user activity, or performing only internal system tasks. Such periods of inactivity may be further distinguished into short time gaps (e.g., on the order of seconds) and long time gaps (e.g., on the order of minutes to hours). Other timing-related events may include, for example, sequences of events occurring in quick succession / bursts of activity.

[0028]

[0042] Exemplary events specific to mobile devices or particularly relevant to mobile device security include screen toggles (on / off), application label / name / icon changes, and screen grabs. Other examples include requests to grant certain types of permissions (e.g., administrator, accessibility), permissions requested dynamically (i.e., during various stages of execution as opposed to installation time), and granting persistence (e.g., foreground services dynamically started by the respective application). Still other examples include attempts to prevent uninstallation of the respective application and displaying overlays on top of the OS settings interface (such overlays may trick unsuspecting users into granting unnecessary permissions for the respective application).

[0029]

[0043] Such event detection may be device-type specific. In one example where client device 12 is a personal or laptop computer, upon detecting the creation of an entity of interest 24, event detector 32 registers the associated set of entities and / or processes with the OS 22's event logging service (e.g., event tracking for Windows®—ETW, Syslog in UNIX®). In response, event detector 32 may receive notification of various events occurring during the execution of each process, either in real time or in log form. Event logging tools typically generate a list of event descriptors that include a timestamp for each event, a numeric code identifying the event type, an indication of the type of process or application that generated each event, and other event parameters. In such an embodiment, detector 32 may detect the occurrence of an event of interest by parsing the respective event logs.

[0030]

[0044] In another example, specialized event detector 32 may modify the set of native functions of OS 22 by inserting redirection instructions (also known as hooks or patches). In this way, when a process executing on client device 12 calls the respective OS function, execution is redirected to a callback routine that notifies detector 32 that the respective OS function is about to be executed. When the hooked function serves a monitored event (e.g., file creation, process launch, etc.), the attempt to call the respective function may serve as an indicator of the occurrence of the respective event.

[0031]

[0045] In yet another example of event detection, electronic communications sent by each client device may be detected by installing a specialized event detector 32 as a proxy module configured to intercept Domain Name Service (DNS) queries and / or HTTP requests transmitted by the client device 12.

[0032]

[0046] Some operating systems, such as those running on smartphones, wearables, etc., may not allow such operations. Nevertheless, other tools may be available to detect the occurrence of various events. For example, some OSes expose application programming interfaces (APIs) that allow registering callbacks for different notifications, inspecting network traffic, detecting SMS / MMS operations, access to storage devices (e.g., SD cards), etc. Some embodiments of event detector 32 use the functionality of accessibility APIs to access on-screen content and detect user interactions with the respective devices and / or applications.

[0033]

[0047] In some embodiments, event detector 32 notifies event dispatcher 36 in response to the occurrence of each event, for example, by transmitting event indicator 35a (FIG. 3). Dispatcher 36 is configured to centralize event notifications from detector 32 and distribute or otherwise make such information accessible to other components of computer security module 30. Dispatcher 36 is further configured to maintain a mapping / association between each detected event and the target software entity that caused the respective event. Data associating each event with the target entity may be provided by event detector 32 and / or event processor 34. In some embodiments, dispatcher 36 stores and / or manages individual events as a data structure that includes fields / attributes that can be strings, integers, Booleans, or bitmaps of flags.

[0034]

[0048] In some embodiments, events may be organized at several semantic levels. Some event detectors 32 may only provide low-level, raw, and / or unstructured data. In some embodiments, a set of event processors 34 is configured to analyze and / or aggregate such primary data to infer the occurrence of higher-level events. Thus, the event processors 34 may receive event indicators 35b via a dispatcher 36 and provide other event notifications 35c to the dispatcher 36 (FIG. 3). In one example, the event processors 34 may add attribute values ​​to events detected by the detectors 32. For example, when the event indicators 35b receive and communicate SMS, an exemplary event processor may determine whether the respective SMS contains a hyperlink and return such information in an updated event indicator 35c. In a more sophisticated example, the event processors 34 may use artificial intelligence (e.g., natural language processing, computer vision, etc.) or other means of analyzing content displayed on a screen to determine whether the entity of interest 24 is displaying a login form, a payment interface, an advertisement, etc. In yet another example, some event processors 34 may combine information about multiple events to determine whether an aggregated higher level event has occurred.

[0035]

[0049] The event processors 34 may be organized in multiple layers, such that the output of one layer further feeds into the event processors of another layer. Such a hierarchical event processing architecture may characterize detected events efficiently and with customizable granularity and complexity. In some embodiments, separate event processing layers may correspond to different event semantic levels. For example, separate processing layers may essentially respond to different questions, such as how, what, and why a target entity performed a particular action. In one such example, the event processing infrastructure of the security module 30 is configured to detect an event involving copying a file to the Windows® startup folder. To perform such an action, the target entity 24 may, for example,

[0036]

[0050] A. Call the Windows(R)API CopyFile function. B. Copy each chunk of the file using a sequence of file read and write commands.

[0037] C. Copy the file using the COM object IFileOperation::Copyitem.

[0051] Event detector 32 may signal the occurrence of low-level events, such as an attempt to execute a CopyFile instruction (case A), an individual file read and / or write (case B), or a COM call to IFileOperation (case C). A set of event processors 34 may consume such low-level events to determine whether they indicate a higher-level file copy event. For example, in case B, some processors 34 may aggregate multiple detected read / write events and determine whether these events involve the same file chunk. If yes, each event processor may transmit an event indicator 35c that notifies the event dispatcher of the occurrence of a file copy event. Another event processor 34 may then collect the file copy events and determine whether they indicate an attempt to copy a file to the startup folder, and if yes, may appropriately generate another event indicator that notifies dispatcher 36.

[0038]

[0052] In some embodiments, to conserve computing resources, some event processors 34 may be selectively activated and / or deactivated, for example, according to the type of entity of interest 24. For example, a selected processor 34 may also be deactivated when other event processors, the feature extractor 42, or the AI ​​module 50 are not currently using the output of the selected processor 34. Some embodiments may use an event processor configuration file 39 to specify interdependencies between event processors 34 and / or to specify which event processors 34 should be active / inactive according to each type of entity of interest. The configuration file 39 may be constructed using any data specification standard known in the art, for example, in a version of Extensible Markup Language (XML) or JavaScript® Object Notation (JSON).

[0039]

[0053] The manner in which event information is dispatched to the event processor 34 and / or other components of the module 30 may vary between embodiments and event types. Exemplary dispatching mechanisms include, for example, fast dispatching, synchronous dispatching, and / or asynchronous dispatching. In fast dispatching, events are submitted directly to the event processor 34 without locking or extra memory allocation. An example includes dispatching data received from a network traffic sensor registered with the Windows® filtering platform. Event processors 34 that collect fast dispatch events are typically pre-registered with the associated event detector and cannot be dynamically activated or deactivated. In synchronous dispatching, the process / thread that generates each event is suspended while the event is submitted to the event processor for analysis and resumed after the analysis is concluded. Thread locking and extra memory allocation further enable the event processor 34 to be dynamically activated / deactivated. In asynchronous dispatching, the process / thread that generates each event is allowed to continue running, and the event notification is added to a queue that is collected by a dedicated processor thread pool. For example, some event processors 34 and / or detectors 32, such as handlers for Event Tracing For Windows (ETW) data, may require asynchronous dispatching.

[0040]

[0054] In some embodiments, the feature extractor 42 determines a set of feature values ​​53 that characterize the behavior of each monitored target entity 24. In some embodiments, as illustrated in FIG. 5, the computer security module 30 maintains a master feature vector 60 for each target software entity 24, the vector 60 comprising a plurality of feature values ​​53 determined by the various feature extractors. The example feature extractors 42a-c of FIG. 5 collectively represent any of the set of feature extractors 42 of FIG. 3. The master feature vector 60 may be initialized to, for example, a set of default feature values ​​or to an empty vector upon startup / creation of each target entity 24. The master feature vector 60 may then evolve over the lifetime of each target entity as each entity performs various actions, causing various events to occur, which are further dispatched to the feature extractor 42 as indicated above and causing updates to the respective feature values ​​53. In some embodiments, the master feature vector 60 includes a relatively large number (e.g., hundreds to thousands) of feature values ​​53. Stated differently, the master vector 60 characterizes each target entity 24 using many attributes. Each feature value 53 is provided by a feature extractor. Nevertheless, not all extractors 42a-c may be active at any time, and thus some feature values ​​53 may be missing or currently pending (discussion in more detail below). In some embodiments, such as those illustrated, a single feature processor may evaluate multiple features / attributes of a target entity and thus provide multiple feature values ​​53 to the master feature vector 60.

[0041]

[0055] To determine dynamic / behavioral features of entities 24, some embodiments of feature extractor 42 evaluate extractor-specific predicates according to events dispatched by event dispatcher 36. A predicate herein represents a statement involving event attributes / fields, such as "flag F is set," "filename ends with .part," "file size > 1 MB," or "IP address is blacklisted." Some predicates may comprise multiple other predicates combined using logical operations such as AND, OR, or XOR. Evaluating a predicate herein involves determining the truth value of the predicate. In a simple example, the selected feature value is equal to the truth value of the respective predicate (e.g., 1 for TRUE, 0 for FALSE).

[0042]

[0056] FIG. 5 further illustrates the relationship between master feature vector 60 and individual AI modules according to some embodiments of the present invention. The set of exemplary AI modules 50a-b collectively represents any of the multiple AI modules 50 of FIG. 3. Each AI module 50a-b receives a respective input feature vector 52a-b. In some embodiments as illustrated, each individual feature vector 52a-b includes only a selected subset of the feature values ​​53 of master feature vector 60, each subset being unique to the respective AI module. This illustrates the intuitive representation that each AI module evaluates target entity 24 from a different “perspective,” i.e., using a distinct but possibly overlapping set of attributes to characterize the target entity. In some embodiments, the size of the individual feature vectors 52a-b is significantly smaller (e.g., dozens to hundreds) than the size of master feature vector 60, which may provide some embodiments with performance advantages as described further below.

[0043]

[0057] In some embodiments, a module activation engine 38 connected to the event dispatcher 36 (FIG. 3) selectively and dynamically activates and / or deactivates AI modules 50 during the lifetime of the target entity 24. Stated differently, the malware detector 44 may effectively use only a limited subset of the available AI modules 50 at any given time, with each module selected according to the static (e.g., entity type) and / or dynamic (i.e., behavior) characteristics of the target entity 24. Activating and deactivating selected AI modules herein refer to adding and removing, respectively, the respective modules to / from a malware detection toolset currently available for use by the detector 44, and do not imply executing the respective modules. As explained in more detail below, the actual execution of an AI module is separate from activation and is triggered by a specific event.

[0044]

[0058] FIG. 6 illustrates an exemplary sequence of steps performed by activation engine 38 according to some embodiments of the present invention. As events are detected, processed, and dispatched to various components, step 602 receives notification of events caused by the execution of target entities 24. Each event may be an entity lifecycle event, such as launching the entity 24, forking a child process, or terminating a process, or a behavioral event, such as opening a file or receiving data via a network adapter. In step 604, engine 38 may identify a set of candidate AI modules 50 for activation according to each event and / or according to the type of target entity. In one such example, some AI modules may be used to detect malicious portable executable files (e.g., Windows® .EXE), while other AI modules may be used to detect malicious Microsoft Word® macros. In another example, a particular type of event may be an indicator of a possible infection by a particular type of malware, such as ransomware. Detecting such an event may therefore prompt engine 38 to select an AI module specialized in ransomware detection for activation. In yet another example, an event involving loading a particular Windows® library may indicate that the target entity 24 may contain Microsoft Word® macros, and therefore prompt the selection of a candidate AI module that specializes in detecting malicious macros.

[0045]

[0059] Some embodiments maintain an association between event types and AI modules potentially activated or deactivated by the occurrence of each type of event, allowing module activation engine 38 to efficiently select candidate AI modules in step 604. The association may be performed in any manner known in the art. In one exemplary embodiment, each AI module 50 may have an associated module configuration file 45 (FIG. 3) that encodes a data structure (e.g., a bitmap) that indicates, among other things, which event types may activate and / or deactivate the respective AI module. Model configuration file 45 may further list the set of entity types that each AI module is trained to monitor.

[0046]

[0060] In response to identifying the candidate modules, activation engine 38 may cycle through each candidate (sequence of steps 606-608). If the current candidate is not already active (step 610 returns NO), then in step 612 engine 38 may evaluate a set of module-specific activation predicates. Such predicates essentially comprise various conditions for activating each AI module and may be included in module configuration file 45. One example of a module activation predicate comprises determining whether a library recently loaded by target entity 24 is for processing Microsoft Word® macros.

[0047]

[0061] When the activation predicate evaluates to true, i.e., when the module activation condition is met (step 614 returns YES), step 616 may mark the respective candidate module as active. In one exemplary embodiment, malware detector 44 may maintain a data structure (e.g., a bitmap) associated with each target entity 24, each data structure indicating which AI modules are currently activated for the respective entity (e.g., each bit in the activation bitmap may correspond to a separate AI module 50). In such an embodiment, step 616 may include toggling a bit corresponding to each candidate in the activation bitmap.

[0048]

[0062] In response to activating each candidate AI module, some embodiments initialize the respective module-specific feature vector (see, e.g., feature vectors 52a-b in FIG. 5). Such initialization may reset any previously determined feature values. In some embodiments, as described further below, activating an AI module further triggers activation of a set of module-specific feature extractors configured to populate the respective module's feature vector.

[0049]

[0063] Some embodiments further allow for deactivating selected AI modules in response to specific events. In one such example, the set of active AI modules includes modules targeted to processes that do not interact with a user. Some embodiments may deactivate the respective modules in response to the targeted entity 24 revealing a user interface.

[0050]

[0064] FIG. 7 shows an example sequence of steps performed by engine 38 to deactivate an AI module. In response to an event notification (step 702), engine 38 may, in step 704, identify an active AI module, e.g., according to an activation bitmap associated with each target entity, as described above. The sequence of steps 706-708 may cycle through all currently active AI modules. Step 710 may evaluate a set of module-specific deactivation predicates, i.e., verify whether a set of conditions for deactivating the selected currently active AI module are met. Such deactivation predicates may, for example, be included in module configuration files 45 associated with each AI module. When the deactivation conditions are met (step 712 returns YES), engine 38 may, in step 714, deactivate the selected AI module. Step 714 may include toggling a bit corresponding to the respective AI module in a module activation bitmap maintained by malware detector 44. In some embodiments, module deactivation may further trigger deactivation of some feature extractors 42 responsible for providing input data to the respective AI module.

[0051]

[0065] FIG. 8 illustrates an exemplary sequence of steps performed by feature extractor 42 to provide input to AI modules 50 according to some embodiments of the present invention. Step 802 receives event notifications dispatched by event dispatcher 36 and identifies the target entities that cause the occurrence of each event. Step 804 may then identify the currently active AI modules associated with each target entity, for example, by examining a module activation bitmap maintained by malware detector 44, as described above. A further sequence of steps 806-808 may cycle through all identified active AI modules. For each active AI module, step 810 may identify a subset of feature extractors that provide input to the respective AI module. In some embodiments, the feature extractors 42 associated with an AI module are specified in the respective AI module's module configuration file 45.

[0052]

[0066] The sequence of steps 812-814 may then cycle through all identified feature extractors. For each feature extractor, step 816 may evaluate a respective predicate according to the event notified in step 802. The predicate may refer to various attributes of the respective event and / or attributes of the target software entity 24 that cause the respective event. In a further step 818, the selected feature extractor 42 may determine feature values ​​53 according to the evaluated predicate and write the values ​​53 to the master feature vector 60 and / or to the module-specific feature vector 52 of the respective AI module.

[0053]

[0067] In one such example of feature extraction, step 802 detects a file read event. Step 810 further identifies feature extractors 42 configured to determine whether the target entity 24 accessed a password file associated with the Chrome® browser from Google, Inc. An exemplary feature extractor predicate may include, for example, determining whether the path of the file accessed by the detected read event matches the location of the Chrome® password file. If YES, each feature extractor may set a respective feature value 53 to "YES" / TRUE.

[0054]

[0068] In some embodiments, an AI module 50 only executes sporadically when triggered by a unique event and / or when a unique execution condition is met. A trigger herein represents a condition for executing a particular AI module. Each condition may be formulated as a set of predicates involving various event attributes and / or attributes of the respective target software entity. A simple exemplary trigger includes the occurrence of a specific type of event, considered herein to be a trigger event. More sophisticated triggers may involve multiple trigger events. In one such example, an AI module executes only when the target entity 24 triggers a first event and also a second event (e.g., when a target process spawns a child and the child process then writes a file to disk). Each trigger event may need to occur in a specific order, etc. While triggers are typically module-specific, in some embodiments, a single trigger may cause multiple AI modules to execute. In some embodiments, the set of optimal trigger events associated with each AI module 50 is determined via a machine learning process, as described below. In response to such training, a computer-readable specification of module-specific triggers, including trigger event types and associated trigger predicates, may be included, for example, in module configuration file 45 (FIG. 3).

[0055]

[0069] In some embodiments, trigger engine 40 receives event notifications from event dispatcher 36 and selectively triggers the execution of various AI modules 50. Figure 9 shows an exemplary sequence of steps performed by trigger engine 40 according to some embodiments of the present invention. Step 902 receives an event indication from dispatcher 36 and appropriately identifies target entities 24. In a further step 904, trigger engine 40 may identify currently active AI modules 50 for each target entity according to, for example, a module activation bitmap maintained by malware detector 44 as described above.

[0056]

[0070] The sequence of steps 906-908 may then cycle through all currently active AI modules. For each active AI module 50, step 910 may retrieve a list of triggers specific to that AI module, for example, by parsing a module configuration file 45 associated with that module. The sequence of steps 912-914 may cycle through the identified triggers. For each such trigger, step 916 may evaluate a trigger-specific predicate according to the trigger specification, according to the event detected in step 902, and / or further according to attributes of the target entity 24. When the trigger predicate evaluates to true (i.e., the condition for triggering the respective AI module is met), step 920 may signal malware detector 44 to execute the respective AI module with the feature data determined for the target entity causing the event detected in step 902.

[0057]

[0071] The trigger engine 40 may then move on to the next active AI module (steps 906-908). In some embodiments as illustrated, multiple AI modules 50 may be triggered by the same event. Also, the same AI module 50 may run multiple times during the target entity's lifetime, triggered, for example, by separate occurrences of the same event or other events listed as triggers for the respective AI module. Nevertheless, by carefully choosing trigger events and AI module parameters via machine learning (see below), some embodiments achieve relatively high malware detection rates and low false positive rates while keeping computational costs low.

[0058]

[0072] In some embodiments, malware detector 44 is configured to selectively execute AI modules 50 in response to a notification from trigger engine 40. FIG. 10 illustrates an exemplary sequence of steps performed by malware detector 44 according to some embodiments of the present invention. In response to a trigger notification (step 1004 returning YES), in step 1006, detector 44 may identify the respective AI modules and target entities according to the trigger notification. A further step 1008 may identify the master feature vector 60 (or module-specific feature vector 52) determined for each target entity. In some embodiments, malware detector 44 may further determine whether all feature values ​​53 required as input by each AI module are available and, if not, may prevent execution of the respective AI module. Alternative embodiments may provide substitute feature values ​​when necessary. Substitutes may be determined according to various methods, such as default values, random values, previous determinations for the same type of target entity, etc.

[0059]

[0073] A further step 1010 may return to executing the respective AI module and listening for trigger notifications (step 1002). The executing may include determining a preliminary verdict 54 (FIG. 3) indicating, for example, the likelihood / probability that the target entity 24 is malicious according to the input feature vector 52. Step 1010 may include, for example, performing a set of tensor multiplication operations with the feature vector 52 and a set of module-specific parameter values ​​(e.g., neural network synaptic weights, convolution masks, etc.), among others. The malware detector 44 may then return to listening for another trigger notification.

[0060]

[0074] In some embodiments, cloud authentication module 46 receives preliminary verdict 54 from malware detector 44. Module 46 may then assemble a final verdict 56 indicating whether target entity 24 is malicious, or more particularly, whether client system 12 contains malware. In determining final verdict 56, module 46 may aggregate multiple individual preliminary verdicts 54 determined by various AI modules 50 and may combine verdict 54 with other malware-indicative data received from other components of computer security module 30 and / or from security server 14. Verdict 56 may then be used by other components of security module 30 to perform malware mitigation actions, such as quarantining target entity 24, erasing malicious software, displaying a warning to the user of client device 12, etc.

[0061]

[0075] In some embodiments, cloud authentication module 46 forwards verdict 54 along with additional data, such as a copy of module-specific feature vector 52 and / or master feature vector 60, to security server 14 (see, e.g., FIG. 1 ). Server 14 may then perform additional malware detection procedures / evaluations and communicate the results of such evaluations back to cloud authentication module 46. For example, server 14 may provide another provisional verdict that may uphold or overturn the verdict 54 received from malware detector 44. In other exemplary embodiments, a record of provisional verdict 54 and associated feature vectors are stored by server 14 for anti-malware research and / or early detection of emerging threats. Such information may be stored in and selectively retrieved from security database 20 ( FIG. 1 ).

[0062]

[0076] Some embodiments may use such collaboration between client device 12 and security server 14 for dynamic and seamless provisioning of new AI modules 50 to client device 12. After new AI modules are developed and trained, they may be deployed to client devices 12a-d and integrated into local instances of malware detector 44 without interrupting the execution of security module 30 or affecting the operation of existing modules. New AI modules may be activated in a provisional “test” state, with any verdicts provided by the new AI modules first being sent to server 14 for validation and then considered in determining final verdict 56. Server 14 may compare each provisional verdict with similar views from other client devices 12a-d to determine, for example, whether the output of the new AI module meets certain performance metrics. A newly installed AI module may be transitioned from the “test” to the “normal” state in response to successfully passing performance tests.

[0063]

[0077] In preparing AI module 50 for use, some embodiments use various machine learning strategies and procedures to train module 50 on a corpus of training software samples, which may include known malicious and / or non-malicious samples. Training typically involves high computational costs and may be performed by a dedicated training system, for example, forming part of security server 14. The AI ​​training system may comprise specialized hardware, such as a dedicated array of processors (e.g., graphics processing units—GPUs) for executing AI module 50.

[0064]

[0078] In one exemplary training scenario, each software sample in the training corpus may be executed in a test environment, such as a set of virtual machines that mimic the hardware and / or software configurations of various client devices 12a-d. The test environment may include instances of event detector 32 and / or event processor 34. However, instead of using an event dispatcher as illustrated above, the test instances of event detector 32 and / or processor 34 may output event indications to an event log, which may be implemented using any data format known in the art (e.g., a JSON file). For each event, the event log may therefore record a set of attribute values ​​that characterize the respective event. Each event log capturing the behavior of the software sample may be further annotated with an expected verdict, for example, indicating whether each training sample is benign or malicious.

[0065]

[0079] In response to determining the corpus of event logs, some embodiments may replay each event log to an AI training system 62, as illustrated in FIG. 11. The components shown in FIG. 11 may be embodied as a computer program executing on, for example, the security server 14. An AI module 150 represents the module currently undergoing training and may collectively represent any of the AI ​​modules 50 of FIG. 3. The AI ​​module 150 may comprise any AI system, such as a set of artificial neural networks having any architecture known in the art. The AI ​​module 150 is configured to receive feature vectors characterizing each software sample and, according to the received feature vectors, determine a preliminary verdict 54 indicating whether each sample is malicious. A set of feature extractors 42 populates the feature vectors with values ​​53 determined according to event specifications read from the event log 70. The feature extractors 42 are described in detail above with respect to FIGS. 3, 5, and 8. A training engine 64 is configured to perform machine learning procedures to train the AI ​​module 150. Training herein involves presenting various training inputs to the AI ​​module 150 and adjusting a set of parameters of the module 150 according to the output each input produces. For each input, some embodiments compute the value of a utility function (sometimes known in the art as a loss) according to each output and / or input, and adjust the module parameters according to the computed utility value with the aim of maximizing (or possibly minimizing) the utility function. In one example where the AI ​​module 150 comprises a neural network, the adjustable parameters may include a set of synaptic weights, while the utility function may quantify the deviation of the current output (provisional verdict 54) from an expected or desired output (e.g., a clean / malicious label associated with each sample / event log 70). Some known variants of training include supervised, unsupervised, self-supervised, and reinforcement learning, among others.In some embodiments, the number of adjustable parameters in a typical AI module 150 may range from hundreds to millions.

[0066]

[0080] FIG. 12 shows an exemplary sequence of steps performed by engine 64 to train AI module 150 according to some embodiments of the present invention. When AI module 150 is first considered, it may be configured to process an initial set of features of the target entity. Each set of features may then be pruned during training to arrive at a final set of features used in production. A sequence of steps 1202-1204 selects the initial set of features and sets up module 150 to use each set of features. AI module 150 is then trained on a corpus of software samples / event logs in step 1206. Training may include, for each training sample / event log 70 in the corpus, determining a feature vector 52, processing each feature vector to produce a verdict 54, determining a utility according to the verdict 54, and adjusting parameters of module 150 to maximize (or minimize) each utility. In some embodiments, the feature vector used in training includes feature values ​​corresponding to the terminal phase of each training sample. An exemplary utility function may quantify the deviation of the verdict 54 from the true verdict (e.g., clean or malicious) for each sample, which may be specified in the event log 70. Step 1206 may be repeated for a predetermined number of training epochs and / or until a performance measure is met.

[0067]

[0081] In some embodiments, the AI ​​module 150 may be further optimized with a process of feature selection, which may be iterative as illustrated. The feature selection process may continue until a termination condition is met (until step 1208 returns YES). Exemplary termination conditions may be formulated according to performance criteria (e.g., malware detection rate, false positive rate, etc.) and further according to the computational cost of running the AI ​​module 150. Some embodiments rely on the observation that both the computational burden and the detection performance of an AI-based detector decrease as the size of the feature vector decreases. Thus, some embodiments strive to achieve a balance in reducing the size of the feature vector without compromising the detection rate.

[0068]

[0082] Feature selection may involve pruning the current feature set, reconfiguring the AI ​​module 150 to use the reduced feature set, and retraining the resulting modified AI module (steps 1210-1214). Step 1210 may implement any feature selection method known in the art, such as discarding a subset of features deemed least reliable in correctly distinguishing malicious from non-infected samples. Such features may be selected according to the statistics of feature values ​​evaluated across the entire training corpus. Other criteria for discarding features may include the computational cost of evaluating each feature, the availability of each feature on the selected device type, etc. Retraining the AI ​​module 150 may proceed similarly to step 1206 described above, even if a different feature vector lacks values ​​for the discarded features.

[0069]

[0083] When feature optimization is complete (step 1208 returns YES), in step 1116, training engine 64 may output module specification 65 comprising a specification of the trained AI module, for example, in the form of a set of optimized module parameter values ​​(e.g., synaptic weights) determined during training. The module specification may further include identifiers of the optimal features determined via feature selection (or alternatively, identifiers of feature extractors 42 for evaluating each feature) and their sequence or location within the feature vector.

[0070]

[0084] In response to training the AI ​​modules 150, some embodiments may proceed to identify a set of trigger events for executing the respective modules. FIG. 13 shows an exemplary sequence of steps performed by the training engine 64 for trigger identification. Step 1302 may identify a training corpus of software samples, e.g., the same corpus used to train the AI ​​modules 150 as described above. The sequence of steps 1304-1306 may then cycle through each training sample. For each training sample embodied by, e.g., the event log 70, step 1308 may initialize a feature vector for the respective AI module. For each event generated by each training sample (steps 1310-1312), step 1314 may identify and execute a set of feature extractors 42 to populate the feature vector, as described above with respect to FIG. 8. Accordingly, step 1314 may include, for example, evaluating extractor-specific predicates according to each event. The sequence of steps 1316-1318 may then execute AI module 150 (already trained as described above), record each verdict, and associate each verdict with the current event selected in step 1312 and the training sample selected in step 1306.

[0071]

[0085] In some embodiments using an event log as described herein, a verdict is computed in the order of occurrence of its respective associated events, thus mimicking the behavior of a malware detector on a client device. By running the trained AI module at various stages during the lifetime of each training sample, some embodiments monitor how the input (feature vector) and output (provisional verdict) of module 150 change in response to various events, which allows an optimal set of triggers to be determined.

[0072]

[0086] By the time all training samples have been processed (step 1304 returns NO), training engine 64 will have amassed a relatively large data set associating potential trigger events with the verdicts produced by AI module 150 in response to each event for each training sample. During the cycle of steps 1320-1322-1324-1326, some embodiments may then select various subsets of trigger candidates from the event-verdict association data set and, for each such subset, compute a detection performance metric for the selected event subset averaged over multiple training samples (e.g., over the entire training corpus).

[0073]

[0087] Various trigger strategies can be optimized in this manner. In a simple example, the illustrated flowchart can determine a single optimal trigger event, i.e., a single optimal stage / moment in the monitored entity's lifetime, such that executing AI module 150 at this particular stage should yield the best chance of correctly detecting malware in the respective entity. More sophisticated trigger selection can comprise selecting multiple optimal triggers. Some embodiments rely on the observation that a particular trigger event may not occur in all instances of the target entity 24, for example, because the respective trigger event may not occur on the type of device on which entity 24 is currently running, or that the respective trigger event may not occur in a particular geographic location, network domain, time interval, etc. Thus, relying on a single type of trigger event may inadvertently result in the failure to trigger the respective AI module. Some embodiments therefore choose multiple optimal triggers, for example, such that the respective AI module is triggered in response to the occurrence of any one of the selected optimal triggers. An additional optimization step can take into account the count of optimal trigger events, relying on the observation that more triggers are likely to improve detection rates, but with the downside of increased computational costs. An optimal trigger set can then be selected that performs satisfactorily across the training corpus while keeping computational costs acceptably low.

[0074]

[0088] In yet another exemplary trigger strategy, some embodiments may choose a set of trigger events such that each AI module is triggered only if all of the respective trigger events occur during execution of the target software entity. Other trigger strategies that may be optimized as described herein include examples in which a set of trigger events must occur in a predetermined order for each AI module to execute. Those skilled in the art will recognize that these trigger choices are exemplary only, and that the present description may be adapted to other options without departing from the scope of the present invention.

[0075]

[0089] In some embodiments, step 1322 selects a subset of candidate trigger events. An exemplary subset consists of a single event. Exemplary measures for selection include the trigger strategy (see discussion above) and the count of training samples representing each candidate trigger. Some embodiments use thresholds to prevent the selection of overly rare and / or overly frequent events as candidate triggers based on the observation that they do not reliably trigger—rare events may not be able to trigger the execution of the respective AI module, and frequent events may not be selective and / or malware-indicative.

[0076]

[0090] A further step 1324 may identify a collection of training samples indicative of the selected candidate trigger event. In the simple example of a single trigger, engine 64 may select all training samples in the corpus that have caused at least one occurrence of the selected candidate trigger. In another example where the trigger strategy includes executing a respective AI module in response to any one of multiple trigger events, engine 64 may select all training samples in the corpus that have caused any one of the selected candidate trigger events. Training samples may further be selected according to other criteria, such as entity type (e.g., portable executable file vs. Microsoft Word® file), device type (e.g., smartphone vs. personal computer), etc.

[0077]

[0091] Step 1326 then determines a performance metric characterizing the selected subset of trigger candidates evaluated / averaged over the selected collection of training samples. Exemplary performance metrics include true detection rate, false positive rate, false negative rate, precision, recall, or a combination thereof. For example, some embodiments may determine the detection performance metric according to an F1 score,

[0078]

number

[0079] where Σ represents the collection of training samples identified in step 1324, the brackets represent evaluating or averaging over Σ, P represents precision (the ratio of correct malicious verdicts out of all malicious verdicts), R represents recall (the ratio of correct malicious verdicts out of all malicious samples), and t p represents the count of true positives (correct malicious verdicts), and f p represents the count of false positives (incorrect malicious verdicts), and f nrepresents the count of false negatives (incorrect non-infected verdicts). All quantities in equation [1] are determined in steps 1316-1318 using the currently selected candidate trigger (step 1322) and the tentative verdicts associated with the training samples (step 1324), and are therefore characteristic of each candidate trigger. Those skilled in the art will recognize that evaluating / averaging a performance metric over a collection of training samples (e.g., Σ in equation [1]) is equivalent to evaluating / averaging each performance metric over the set of tentative verdicts associated with each sample.

[0080]

[0092] When the termination condition for selecting trigger candidates is met (step 1320 returns YES), in step 1328, training engine 64 may select an optimal trigger set according to the detection performance of the various candidate trigger subsets evaluated in steps 1320-1326. For example, some embodiments may select as the optimal trigger the subset of trigger candidates that exhibits the best detection performance (e.g., highest F1 score) across the training corpus.

[0081]

[0093] A further step 1330 may output a trigger specification 67 (FIG. 11) comprising a specification of the identified optimal trigger events for use by trigger engine 40 as described above with respect to FIG. 9. In some embodiments, trigger specification 67 further includes a set of trigger predicates, each predicate comprising a condition for triggering a respective AI module in response to detecting a respective trigger event (e.g., step 916 of FIG. 9). In some embodiments, module specification 65 and trigger specification 67 form part of module configuration file 45 provided to protected client devices 12a-d. Step 1330 may thus be considered to configure malware detector 44. Such configuration may be performed dynamically, as trigger specification 67 may be kept up to date and distributed to clients independently of other components of security module 30.

[0082]

[0094] FIG. 14 illustrates an exemplary hardware configuration of a computer system 80 programmed to execute portions of the methods described herein. The computer system 80 may represent any of the client devices 12a-d, the security server 14, and the AI ​​training system 62. The illustrated computing appliance is a personal computer; other devices, such as servers, mobile phones, tablet computers, and wearables, may have slightly different configurations. The processor 82 comprises a physical device (e.g., a microprocessor, a multi-core integrated circuit formed on a semiconductor substrate) configured to perform computational and / or logical operations on sets of signals and / or data. Such signals or data may be encoded in the form of processor instructions, e.g., machine language, and delivered to the processor 82.

[0083]

[0095] Processors 82 are generally characterized by an instruction set architecture (ISA), which specifies, among other things, the respective set of processor instructions (e.g., x86 family vs. ARM® family) and the size of registers (e.g., 32-bit vs. 64-bit processors). The architecture of processor 82 may further vary according to its intended primary usage. Central processing units (CPUs) are general-purpose processors, while graphics processing units (GPUs) may be optimized for image / video processing and some forms of parallel computing. Processors 82 may further include application-specific integrated circuits (ASICs), such as tensor processing units (TPUs) from Google®, Inc., and neural processing units (NPUs) from various manufacturers. TPUs and NPUs may be particularly well-suited for machine learning applications such as those described herein.

[0084]

[0096] The memory unit 84 may comprise a volatile computer-readable medium (e.g., dynamic random access memory—DRAM) that stores data / signal / instruction encodings accessed or generated by the processor 82 while performing operations. The input devices 86 may include, among other things, a computer keyboard, a mouse, and a microphone, including respective hardware interfaces and / or adapters that allow a user to introduce data and / or instructions into the computer system 80. The output devices 88 may include, among other things, display devices such as a monitor and speakers, and hardware interfaces / adapters such as a graphics card that allow the respective computing appliance to communicate data to a user. In some embodiments, the input / output devices 86-88 share common hardware (e.g., a touchscreen). The storage device 92 comprises a computer-readable medium that allows for the non-volatile storage, reading, and writing of software instructions and / or data. Exemplary storage devices include magnetic and optical disks and flash memory devices, as well as removable media such as CD and / or DVD disks and drives. Network adapter 94 enables computer system 80 to connect to electronic communications networks (eg, network 15 in FIG. 1) and / or to other devices / computer systems.

[0085]

[0097] Controller hub 90 generally represents multiple system, peripheral, and / or chipset buses and / or all other circuitry that enables communication between processor 82 and the remaining hardware components of computer system 80. For example, controller hub 90 may include a memory controller, an input / output (I / O) controller, and an interrupt controller. Depending on the hardware manufacturer, some such controllers may be incorporated into a single integrated circuit and / or may be integrated with processor 82. In another example, controller hub 90 may include a northbridge that connects processor 82 to memory 84 and / or a southbridge that connects processor 82 to devices 86, 88, 92, and 94.

[0086]

[0098] The exemplary systems and methods described above enable efficient detection of malicious software using artificial intelligence (AI). Traditional anti-malware software typically uses a set of heuristics (i.e., simple rules) to determine whether a software entity is malicious according to various data describing the behavior of each entity. For example, the security software may detect a set of hardware and / or software events caused by the execution of the target entity and look for behavioral patterns commonly associated with malware. Some such events may themselves indicate malware, while others, when occurring together with other events, indicate malicious intent.

[0087]

[0099] A few modern security solutions use AI techniques (e.g., trained artificial neural networks) to detect malware. In some such examples, various characteristics of target software are presented as inputs to a neural network classifier that has been pre-trained to distinguish between clean and malicious software samples. Despite its many advantages, AI techniques come with their own significant technical challenges. For example, a typical neural network classifier can be 10 times more computationally expensive than a simple set of heuristic methods. Another challenge is input selection, as not all attributes of a target entity are equally informative in identifying malware. Furthermore, not all selected attributes may be available at all times and on all devices. Some attributes of software entities may be significantly more expensive to evaluate than others. Yet another challenge is the extreme diversity and speed of evolution of malicious software, potentially rendering any current selection of input features obsolete despite the threats posed.

[0088]

[0100] One conventional solution to these challenges involves increasing the size of the neural network and associated input vector. Some such classifiers end up using thousands of distinct attributes of the target entity. Yet, training and running classifiers of such large size imposes significant computational costs, making such solutions impractical for client devices with relatively limited resources, such as smartphones and wearables. Large-scale classifiers are also difficult to update, as updating essentially involves retraining and distributing the entire retrained classifier to the client.

[0089]

[0101] Some embodiments of the present invention directly address some of these challenges. As opposed to using large-scale classifiers and feature vectors, some embodiments employ relatively small AI modules. To combat the performance disadvantage that plagues small neural networks, some embodiments use numerous (e.g., tens or hundreds) of distinct AI modules to provide sophisticated, multifaceted views of the target entity. In one such example, each AI module 50 (FIG. 3) may process a distinct set of attributes / features of the target software. Some embodiments rely on the observation that the computational costs associated with neural computations increase non-linearly with the size of the associated neural network; thus, running numerous small networks is typically less costly than running a single large network.

[0090]

[0102] Some embodiments rely on further optimization strategies to reduce the computational overhead of security software. For example, only a selected, relatively small subset of available AI modules is activated (i.e., ready to run) when monitoring each entity of interest. The selection of active AI modules may vary depending on the type of entity of interest (e.g., portable executable file vs. Microsoft Word® file), the type of client device (e.g., desktop computer vs. smartphone), or the current state or load of each client device, among other factors.

[0091]

[0103] Furthermore, in some embodiments, the composition of the security toolkit (i.e., the count and selection of AI modules currently in use) changes dynamically while the target software is running. To conserve memory and processor cycles, some AI modules may be turned off and back on according to behavioral cues, such as specific events triggered by the target entity. In one such example, an initial set of AI modules is activated upon startup of the target entity. In response to specific actions of the target entity, some of the initial AI modules may be deactivated and others may be added to the toolkit.

[0092]

[0104] Yet another optimization feature of some embodiments of the present invention includes sporadically executing only currently active AI modules in response to unique events herein considered to be trigger events. Some embodiments rely on the observation that feature vectors may evolve during the lifetime of a monitored entity as each entity performs its activities; therefore, inputs to an AI module may not be equally indicative of malware at all times. In other words, there may be an optimal stage in the lifetime of a monitored entity (e.g., identified by a trigger event) during which executing an AI module is optimally rich in information about the malicious intent of each entity. Such an optimal trigger may provide, for example, a trade-off between early detection and reliable detection.

[0093]

[0105] Some embodiments use a machine learning procedure to identify an optimal trigger by measuring the malware detection performance of a trained AI module at each of a set of candidate trigger points, averaged over a corpus of training samples as detailed above. Overall, the procedure may identify multiple optimal trigger events. In some embodiments, having more than one trigger event means that the respective AI module is triggered by the occurrence of any one of the respective trigger events (i.e., alternate triggers). In other embodiments, multiple triggers may dictate that the respective AI module executes only if all of the respective trigger events occur. Some embodiments may further dictate the order in which the trigger events must occur so that the respective AI module executes. Triggers may be specific to each AI module and further specific to the type of software entity of interest, device type, etc.

[0094]

[0106] The modular architecture of security software as described herein further facilitates rapid and seamless upgrades by incorporating newly developed AI modules and discarding older, less powerful modules without potentially affecting the operation of other components of the security software. Such flexibility enables some embodiments to maintain a sufficient level of detection performance despite new and emerging threats, while significantly reducing time to market by avoiding substantial overhauls of current security solutions and expensive retraining of large-scale NN detectors. In one exemplary embodiment, new or upgraded AI modules can be deployed to client devices at any time in the form of module-specific configuration files containing specifications for each AI module (e.g., a set of module parameter values, such as synaptic weights, resulting from training) and a set of trigger events that cause the selective execution of each AI module. A malware detector can instantiate each AI module as dictated by the module specification, dynamically insert each AI module into the current toolkit, and selectively trigger the execution of each AI module as dictated by the trigger specification. The same dynamic deployment strategy can apply to other aspects of the described security solutions. For example, the set of triggers and / or conditions for activating and deactivating deployed AI modules can be updated on the fly and distributed to clients without affecting other components of the security software.

[0095]

[0107] It will be apparent to those skilled in the art that the above embodiments can be varied in many ways without departing from the scope of the present invention. Accordingly, the scope of the present invention should be determined by the following claims and their legal equivalents.

Claims

1. determining a trigger event for executing a malware detection artificial intelligence (AI) module, wherein determining the trigger event includes: determining a plurality of preliminary verdicts indicating whether a training software entity is malicious, each verdict of the plurality of preliminary verdicts being associated with a distinct event of a plurality of events caused by execution of the training software entity, and determining each verdict includes: updating a feature vector characterizing the training software entity according to the distinct events; executing the AI ​​module to determine a respective verdict according to the updated feature vector; determining a number of preliminary verdicts, including: in response to determining the plurality of preliminary verdicts, selecting the trigger event from the plurality of events in accordance with the plurality of preliminary verdicts; determining a trigger event, configuring the malware detector to selectively execute another instance of the AI ​​module in response to an occurrence of the trigger event to determine whether the target software entity is malicious in response to determining the trigger event; 1. A computer system comprising at least one hardware processor configured to perform:

2. 2. The computer system of claim 1, wherein the at least one hardware processor: determining another trigger event for executing the AI ​​module by selecting the another trigger event from the plurality of events according to the plurality of preliminary verdicts; configuring the malware detector to selectively execute the separate instance of the AI ​​module in response to the occurrence of the separate trigger event to determine whether the target software entity is malicious; The computer system is further configured to:

3. 2. The computer system of claim 1, wherein the at least one hardware processor is configured to determine each verdict of the plurality of provisional verdicts according to an order of occurrence of the plurality of events during the execution of the training software entity.

4. 2. The computer system of claim 1, wherein determining the trigger event comprises: determining a malware detection performance of the trigger event across a collection of training software entities, the malware detection performance being determined according to another plurality of provisional verdicts, each verdict of the another plurality of provisional verdicts indicating whether a respective entity of the collection is malicious, each verdict of the another plurality of provisional verdicts being associated with an occurrence of the trigger event during execution of the respective entity; selecting the trigger event from the plurality of training events further according to the malware detection performance of the trigger event; The computer system further comprises:

5. 5. The computer system of claim 4, wherein the at least one hardware processor is configured to determine the malware detection performance of the trigger event according to a count of true positive verdicts within the other plurality of provisional verdicts.

6. 6. The computer system of claim 5, wherein the at least one hardware processor: [Equation 1] and configured to determine the malware detection performance of the trigger event according to t ｐ represents the count of true positive verdicts, and f ｐ represents the count of false positive verdicts, and f ｎ represents a count of false negative verdicts within the other plurality of provisional verdicts.

7. 5. The computer system of claim 4, wherein the at least one hardware processor is configured to determine the malware detection performance of the trigger event according to a count of false positive verdicts within the other plurality of provisional verdicts and further according to a count of false negative verdicts within the other plurality of provisional verdicts.

8. 2. The computer system of claim 1, wherein configuring the malware detector includes assembling a specification of the trigger event and transmitting the specification of the trigger event to a client device running the malware detector.

9. 9. The computer system of claim 8, wherein the specification of the trigger event further includes a specification of a trigger predicate comprising a condition for executing the other instance of the AI ​​module in response to the occurrence of the trigger event.

10. 2. The computer system of claim 1, wherein the malware detector comprises a plurality of detection modules including the separate instances of the AI ​​module, and wherein selectively executing the separate instances of the AI ​​module comprises selecting the separate instances of the AI ​​module for execution from the plurality of detection modules in response to the occurrence of the trigger event.

11. using at least one hardware processor of a computer system, determining a trigger event for executing a malware detection artificial intelligence (AI) module, the determining the trigger event comprising: determining a plurality of preliminary verdicts indicating whether a training software entity is malicious, each verdict of the plurality of preliminary verdicts being associated with a distinct event of a plurality of events caused by execution of the training software entity, the step of determining each verdict comprising: updating a feature vector characterizing the training software entity according to the distinct events; running the AI ​​module to determine a respective verdict according to the updated feature vector; determining a plurality of preliminary verdicts, including: in response to determining the plurality of preliminary verdicts, selecting the trigger event from the plurality of events in accordance with the plurality of preliminary verdicts; determining a trigger event, configuring the malware detector to selectively execute another instance of the AI ​​module in response to the occurrence of the trigger event to determine whether the target software entity is malicious in response to determining the trigger event; A computer security method comprising:

12. 12. The method of claim 11, further comprising using the at least one hardware processor: determining another trigger event for executing the AI ​​module by selecting the another trigger event from the plurality of events according to the plurality of preliminary verdicts; configuring the malware detector to selectively execute the other instance of the AI ​​module in response to the occurrence of the other trigger event to determine whether the target software entity is malicious; The method further comprises:

13. 12. The method of claim 11, comprising determining each verdict of the plurality of preliminary verdicts according to an order of occurrence of the plurality of events during the execution of the training software entity.

14. 12. The method of claim 11, wherein determining the trigger event comprises: determining a malware detection performance of the trigger event across a collection of training software entities, the malware detection performance being determined according to another plurality of provisional verdicts, each verdict of the another plurality of provisional verdicts indicating whether a respective entity of the collection is malicious, and each verdict of the another plurality of provisional verdicts being associated with an occurrence of the trigger event during execution of the respective entity; selecting the trigger event from the plurality of training events further according to the malware detection performance of the trigger event; The method further comprises:

15. 15. The method of claim 14, comprising determining the malware detection performance of the trigger event according to a count of true positive verdicts within the other plurality of provisional verdicts.

16. 16. The method of claim 15, [Equation 2] determining the malware detection performance of the trigger event according to t ｐ represents the count of true positive verdicts, and f ｐ represents the count of false positive verdicts, and f ｎ represents the count of false negative verdicts within said another plurality of provisional verdicts.

17. 15. The method of claim 14, comprising determining the malware detection performance of the trigger event according to a count of false positive verdicts within the other plurality of provisional verdicts and further according to a count of false negative verdicts within the other plurality of provisional verdicts.

18. 12. The method of claim 11, wherein configuring the malware detector comprises assembling a specification of the trigger event and transmitting the specification of the trigger event to a client device running the malware detector.

19. 20. The method of claim 18, wherein the specification of the trigger event further includes a specification of a trigger predicate comprising a condition for executing the further instance of the AI ​​module in response to the occurrence of the trigger event.

20. 12. The method of claim 11, wherein the malware detector comprises a plurality of detection modules including the separate instances of the AI ​​module, and wherein selectively executing the separate instances of the AI ​​module comprises selecting the separate instances of the AI ​​module for execution from the plurality of detection modules in response to the occurrence of the trigger event.

21. When executed by at least one hardware processor of a computer system, determining a trigger event for executing a malware detection artificial intelligence (AI) module, wherein determining the trigger event includes: determining a plurality of preliminary verdicts indicating whether a training software entity is malicious, each verdict of the plurality of preliminary verdicts being associated with a distinct event of a plurality of events caused by execution of the training software entity, and determining each verdict includes: updating a feature vector characterizing the training software entity according to the distinct events; executing the AI ​​module to determine a respective verdict according to the updated feature vector; determining a number of preliminary verdicts, including: in response to determining the plurality of preliminary verdicts, selecting the trigger event from the plurality of events in accordance with the plurality of preliminary verdicts; determining a trigger event, configuring the malware detector to selectively execute another instance of the AI ​​module in response to an occurrence of the trigger event to determine whether the target software entity is malicious in response to determining the trigger event; A non-transitory computer-readable medium storing instructions for causing the computer system to perform the above.

Images