Aggregated Event Profiles for Detecting Malicious Mobile Applications

Abstract

The described systems and methods protect client devices, such as personal computers and IoT devices, from malicious software. In some embodiments, multiple client devices report the occurrence of various events to a security server, with each such event being caused by a local instance of a target application (e.g., a mobile app) running on the respective device. The security server then collates the behavior of each target application across the multiple client devices. In some embodiments, an aggregate event set and / or sequence is calculated that combines events detected on one device with events detected on other devices, and the aggregate event set / sequence determines whether the target application is malicious.

Description

[Technical Field]

[0001]

[0001] The present invention relates to computer security, and more particularly to protecting users and devices from malicious software. [Background technology]

[0002] Malicious software, also known as malware, affects numerous computer systems worldwide. Malware, in its many forms—including computer viruses, spyware, and ransomware—poses serious risks to millions of computer users, making them particularly vulnerable to data and confidential information loss, identity theft, and lost productivity. The explosive growth of mobile computing further exacerbates computer security risks, with millions of devices, such as smartphones and tablet computers, constantly connected to the Internet and becoming potential targets for malware. Particularly on mobile platforms, malware can pose as legitimate applications, such as games, news, or messaging platforms, tricking unsuspecting users into installing the malware. In some cases, users may be completely unaware of malicious activity taking place beneath a seemingly legitimate and engaging interface.

[0003]

[0003] Traditional methods of malware protection include running security software on each device. Detection methods typically include static analysis, in which the target software is compared to a library of "code signatures" indicative of malware, and behavioral analysis, in which the security software monitors the target software for signs of malicious behavior. Once the security software detects a malicious agent, it can further quarantine or prevent its execution.

[0004] Sophisticated malware can often evade such countermeasures. Some malware refrains from exhibiting malware-indicative behavior for long periods of time, fooling security software into classifying the malware as benign. Some malware may adapt its behavior according to device type (e.g., smartphone vs. tablet, one manufacturer or model vs. another), operating system type, each device's current geographic location, etc. Some malware further selects victims by searching each device (e.g., smartphone) for indicators that indicate the user's value to the attacker. For example, malware may determine other software currently installed on each device, searching for specific applications such as banking or social networking sites. Other malware may monitor user access patterns for various applications, online resources, etc. Such malware may launch attacks (e.g., a series of malicious actions) only against carefully selected devices and / or carefully selected users, at times when each attack is deemed likely to be successful.

[0005]

[0005] When attacks occur on only a small fraction of devices infected with a malicious software agent, traditional security software can have difficulty protecting individual users and devices. Workarounds such as those described above substantially complicate the observation of malicious behavior in applications "in the wild," thereby hindering the development of behavioral signatures that can be used to classify individual applications as malicious. Meanwhile, the lack of behavioral signatures for detecting malicious agents potentially leaves a large number of users unprotected. Summary of the Invention [Problem to be solved by the invention]

[0006]

[0006] Therefore, there is great interest in developing computer security systems and methods that can quickly and efficiently respond to new threats to mobile computing platforms. [Means for solving the problem]

[0007] According to one aspect, a method for protecting a plurality of client devices from malware includes using at least one hardware processor of a computer system to select a plurality of events from an event pool according to a cause of the plurality of events, wherein the selected plurality of events are all caused by a target software application. A first event of the plurality of events is caused by one instance of the target application running on one of the plurality of client devices, and a second event of the plurality of events is caused by another instance of the target application running on another of the plurality of client devices. The method further includes using the at least one hardware processor to arrange the selected plurality of events according to a time of occurrence of each event to form an aggregate event sequence including the first event and the second event. The method further includes using the at least one hardware processor to determine whether the target application is malicious according to the aggregate event sequence.

[0008] According to another aspect, a computer system includes at least one hardware processor configured to select a plurality of events from an event pool according to a cause of the plurality of events, the selected plurality of events all being caused by a target software application. A first event of the plurality of events is caused by one instance of the target application running on one of the plurality of client devices, and a second event of the plurality of events is caused by another instance of the target application running on another of the plurality of client devices. The at least one hardware processor is further configured to arrange the selected plurality of events according to a time of occurrence of each event to form an aggregate event sequence including the first event and the second event. The at least one hardware processor is further configured to determine whether the target application is malicious according to the aggregate event sequence.

[0009] According to another aspect, a non-transitory computer-readable medium stores instructions that, when executed by at least one hardware processor of a computer system, cause the computer system to select a plurality of events from an event pool according to causes of the plurality of events, wherein the selected plurality of events are all caused by a target software application. A first event of the plurality of events is caused by one instance of the target application running on one of the plurality of client devices, and a second event of the plurality of events is caused by another instance of the target application running on another of the plurality of client devices. The instructions further cause the computer system to arrange the selected plurality of events according to a time of occurrence of each event to form an aggregate event sequence including the first event and the second event. The instructions further cause the computer system to determine whether the target application is malicious according to the aggregate event sequence.

[0010]

[0010] The above aspects and advantages of the present invention will be better understood upon reading the following detailed description and upon reference to the drawings, in which: [Brief explanation of the drawings]

[0011] [Figure 1]

[0011] FIG. 1 illustrates an exemplary set of client devices protected from malicious software in accordance with some embodiments of the present invention. [Figure 2]

[0012] FIG. 2 illustrates a typical data exchange between a client device and a security server, according to some embodiments of the present invention. [Figure 3]

[0013] FIG. 2 illustrates exemplary components executing on a client device in accordance with some embodiments of the present invention. [Figure 4]

[0014] FIG. 2 illustrates exemplary components executing on a security server according to some embodiments of the present invention. [Figure 5]

[0015] FIG. 2 illustrates an exemplary sequence of steps performed by a security server according to some embodiments of the present invention. [Figure 6-A]

[0016] FIG. 2 illustrates an exemplary sequence of steps performed by a behavior analyzer according to some embodiments of the present invention. [Figure 6-B]

[0017] FIG. 4 illustrates another exemplary sequence of steps performed by a behavior analyzer according to some embodiments of the present invention. [Figure 7-A]

[0018] 1A-1C illustrate exemplary aggregated event sequences assembled from events received from multiple client devices and exemplary behavioral signatures extracted from each aggregated event sequence, according to some embodiments of the present invention. [Figure 7-B]

[0019] FIG. 10 illustrates another exemplary aggregate event sequence assembled from events received from multiple client devices, according to some embodiments of the present invention. [Figure 7-C]

[0020] FIG. 10 illustrates yet another exemplary aggregate event sequence assembled from events received from multiple client devices, according to some embodiments of the present invention. [Figure 8]

[0021] FIG. 2 illustrates an exemplary aggregated event sequence annotated with various event metadata, according to some embodiments of the present invention. [Figure 9]

[0022] FIG. 1 illustrates an exemplary hardware configuration for a computer system programmed to perform some of the methods described herein. DETAILED DESCRIPTION OF THE INVENTION

[0012]

[0023] In the following description, it should be understood that all referenced connections between structures may be direct operational connections or indirect operational connections via intermediate structures. A set of elements includes one or more elements. Any description of an element should be understood to refer to at least one element. A plurality of elements includes at least two elements. Unless otherwise required, any described method steps do not necessarily have to be performed in the particular order shown. A first element (e.g., data) derived from a second element encompasses a first element equal to the second element and a first element generated by processing the second element and, optionally, other data. Making a decision or determination according to a parameter encompasses making a decision or determination according to the parameter and, optionally, other data. Unless otherwise specified, an indicator of a quantity / data may be the quantity / data itself or may be an indicator different from the quantity / data itself. A computer program is a sequence of processor instructions that perform a task. The computer programs described in some embodiments of the present invention may be standalone software entities or subentities (e.g., subroutines, libraries) of other computer programs. The term "database" is used herein to refer to any organized collection of data. A hash is the numerical result of applying a hash function to a token (e.g., a string, a code snippet, etc.). A hash function maps data of arbitrary size to a value of fixed size. Exemplary hashing functions / procedures include cyclic redundancy checks (CRCs), checksums, message digest functions (e.g., MD5), and secure hash algorithms (SHAs), etc. Computer-readable media encompass non-transitory media such as magnetic, optical, and semiconductor storage media (e.g., hard drives, optical disks, flash memory, DRAM), as well as communication links such as conductive cables and fiber optic links.According to some embodiments, the present invention provides, inter alia, computer systems comprising hardware (e.g., one or more processors) programmed to perform the methods described herein, as well as computer-readable media encoding instructions for performing the methods described herein.

[0013]

[0024] The following description illustrates embodiments of the present invention by way of example, and not necessarily by way of limitation.

[0025] FIG. 1 illustrates a system 10 for protecting a set of client devices 12a-d from malicious software in accordance with some embodiments of the present invention. Client devices 12a-d generally represent any electronic device having a processor, memory, and a communication interface that enables each device to communicate with other devices / computer systems. Exemplary client devices 12a-d include personal computers, mobile computing platforms (e.g., laptop computers, tablets, mobile phones), entertainment devices (e.g., televisions, game consoles), wearable devices (e.g., smart watches, fitness bands), and home appliances (e.g., refrigerators, washing machines). In the exemplary configuration of FIG. 1, client devices 12a-d are connected to a communications network 15, which may include a local area network (e.g., a home network, an enterprise network, etc.), a wide area network, and / or the Internet. Network 15 generally represents a set of hardware (physical layer) and software interfaces that enable the transfer of data between devices 12a-d and other entities connected to network 15. Client devices 12a-d can run various software, such as internet browsing, word processing, games, electronic messaging, social networking applications, etc., and exchange data with other computer systems (e.g., remote servers) over network 15.

[0014]

[0026] FIG. 1 also shows a security server 14 connected to communications network 15. Server 14 generally represents a set of communicatively coupled computer systems, which may or may not be physically proximate to one another. In some embodiments, server 14 protects client devices 12a-d by analyzing the behavior of various software applications executing on each client device to determine whether each application is engaging in malicious activity, such as fraud or theft of sensitive data, among other things. In some embodiments, server 14 interfaces with a security database 20 containing an organized collection of computer security data. For example, database 20 may store a collection of reference hashes that can be used to evaluate the integrity of applications executing on protected client devices 12a-d and / or a set of code signatures that can be used to determine whether selected applications contain malicious software. In yet another example, security database 20 may store a collection of behavioral signatures that characterize a set of malicious and / or benign software applications, which allows server 14 to determine whether selected applications executing on protected client devices 12a-d are malicious. In some embodiments, database 20 may further store a set of event indicators that indicate various events occurring on client devices 12a-d, which events are caused by the execution of various software applications, as described in more detail below.

[0015]

[0027] In some embodiments, database 20 may further store multiple client records associated with client devices 12a-d and / or users of each client device. In one example, each client record corresponds to a distinct client device 12a-d. The client records may store a set of identifiers for each client device (e.g., Media Access Control (MAC) address, International Mobile Equipment Identity (IMEI) number, network / IP address, etc.). Another exemplary client identifier may include a device-specific token (e.g., a hash) uniquely associated with an instance of a security application executing on each client device. The operation of the security application is described in more detail below.

[0016]

[0028] In some embodiments, the client record may further include a device profile for each client device 12a-d, including a set of hardware and / or software characteristics of the respective device. An exemplary device profile may include an indication of the device type (e.g., smartphone, smartwatch, tablet computer, etc.), an indication of the device model / version (e.g., iPhone® 12, Samsung® Galaxy Watch® 4, etc.), and an indication of the operating system type and / or version (e.g., iOS® 15.5, Android® 12, etc.). Other device profile data may include a list of applications currently installed on the respective device, an indication of the respective device's physical location (e.g., a set of GPS coordinates, a country / city / region indication, etc.), and a set of current values ​​for various operating system settings for the respective device, etc. In some embodiments, the device profile data may include user profile data that selectively places users of each client device into user categories, for example, according to typical usage (e.g., business, personal), personal interests (e.g., gaming, shopping, social networking), etc. Such user profile data may be determined, for example, according to each user's online habits, or by any other method known in the art.

[0017]

[0029] Database 20 may be formatted and stored according to any standard known in the art. Exemplary database formats include relational databases, Extensible Markup Language (XML) databases, spreadsheets, key-value stores, etc. Server 14 may be configured to selectively retrieve and / or insert data in database 15 using, for example, structured queries.

[0018]

[0030] FIG. 2 illustrates an exemplary data exchange between a client device 12 and a security server 14, according to some embodiments of the present invention. Device 12 generally represents any of client devices 12a-d of FIG. 1. In some embodiments, security software executing on device 12 is configured to detect the occurrence of events caused by, or otherwise directly related to, the execution of selected software applications on the respective device and to transmit event indicators 16 describing the respective events to security server 14. The exemplary event indicators 16 may include an indicator of the event type of the respective event and may further include a timestamp indicating the time of occurrence of the respective event. In some embodiments, event indicator 16 may further include an identifier of client device 12, e.g., an ID tag associated with each instance of a security application, allowing server 14 to unambiguously associate each event with device 12. Event indicator 16 may further include an identifier (e.g., an integrity hash) of the application that executed and caused the respective event.

[0019]

[0031] In some embodiments, security software executing on device 12 may be further configured to determine various device profile data 17 characterizing each device and / or user and transmit the profile data 17 to server 14. Exemplary device profile data includes the make and model of device 12, geolocation indicators, etc. (e.g., see the device profile description above).

[0020]

[0032] In some embodiments, the server 14 may further transmit a security notification 18 to the client device 12, the notification 18 indicating, for example, that a selected application installed on the device 12 and / or a selected application currently running on the device 12 is malicious. In response to receiving the notification 18, security software running on the device 12 may display a warning to a user of the device 12 and may further block execution of the respective malicious application or partially disable the functionality of the application. The notification 18 may further include an indication of whether the respective instance of the application actually performed malicious activity on the respective device and / or data that enables the local security software to display a set of instructions for mitigating the effects of the respective malware. An exemplary warning displayed to the user may include a hyperlink to an online description of the respective malware agent and instructions for recovering from its effects (e.g., instructions for removing the respective malicious application, changing various operating system settings, etc.).

[0021]

[0033] 3 illustrates exemplary components of client device 12, according to some embodiments of the present invention. Some or all of the illustrated components may be implemented as software executing on a hardware processor of client device 12. However, those skilled in the art will recognize that some of the respective functions may be implemented in dedicated hardware and / or a combination of hardware and software. An operating system (OS) 22 provides an interface between the hardware of client device 12 and other computer programs, such as target applications 24, executing on each client device. Exemplary operating systems include Windows®, MacOS®, iOS®, and Android®, among others.

[0022]

[0034] The target application 24 generally represents any software application, such as a word processor, image processing, spreadsheet, calendar, game, social networking site, web browser, and electronic communication application. The term "application" herein refers to independent executable software, distinct from an operating system, that can be invoked by a user independently of other applications (e.g., as opposed to a software library or subroutine that forms part of another computer program). An exemplary application 24 includes a smartphone app downloaded from a store such as the App Store by Apple®, Inc. or Google® Play®. Those skilled in the art will recognize that the inclusion of only one target application 24 in FIG. 3 is not intended to be limiting. Client devices 12, such as smartphones and personal computers, typically have dozens of installed applications. The "target" modifier is included merely to clarify that the application 24 forms the subject of the security analysis (malware detection) described herein. However, such language does not limit the analysis to only one target application; in some embodiments, the behavior of multiple applications can be monitored simultaneously.

[0023]

[0035] Additionally, this disclosure uses the term "target application" to refer to all identical instances of a respective application, regardless of the device on which each instance is running. As described in more detail below, some embodiments collate events caused by target applications 24 across multiple devices, for example, to construct aggregate event sequences. Such language should not be construed to indicate that all such events are caused by a single instance of the respective application; rather, different events may be caused by different instances of application 24. Thus, a "malicious" or "clean" determination made for application 24 applies to all identical instances of the respective application.

[0024]

[0036] In some embodiments, security application 30 is configured to cooperate with security server 14 to determine whether target application 24 poses a computer security risk to the user. Security application 30 may form part of a larger software suite that may provide various security services, such as traffic filtering, virtual private networking (VPN), anti-spam, parental control, etc. In some embodiments, security application 30 further includes an event harvester module 32 configured to detect the occurrence of various software and / or hardware events caused by the execution of target application 24 and to send a computer-readable account of each event to security server 14 in the form of event indicators 16. Detected events may or may not be indicative of malware by themselves; as will be described in further detail below, some events may be indicative of malware when occurring together with other events and / or in certain sequences.

[0025]

[0037] Exemplary detected events include application installation, uninstallation, and updates, process / application launch and termination, child process spawning (e.g., forking), dynamic library loading / unloading, execution of specific processor instructions (e.g., system calls), file events such as file creation, writing, and deletion, and various OS parameter settings (e.g., Windows® registry events, permission / privilege changes), etc. Other exemplary events include requests to access peripheral devices (e.g., hard disks, SD cards, network adapters, microphones, cameras), requests to access remote resources (e.g., Hypertext Transfer Protocol (HTTP) requests to access specific URLs, attempts to access document repositories over a local network), requests formulated with specific uniform resource identifier schemes (e.g., mailto: or ftp: requests, etc.), and attempts to send electronic messages (e.g., email, Short Message Service (SMS), etc.). Still other exemplary events include moving the user interface / window of the target application 24 in and / or out of focus / foreground.

[0026]

[0038] Some embodiments can detect various timing-related events, such as periods of inactivity, i.e., time gaps between events and / or time intervals during which the respective client device is idle, does not register user activity, or performs only internal system tasks. Such periods of inactivity may be further distinguished into short time gaps (e.g., on the order of seconds) and long time gaps (e.g., on the order of minutes to hours). Other timing-related events may include, for example, a series of events / bursts of activity occurring in succession.

[0027]

[0039] Other example detected events may include receiving and / or displaying certain types of content, such as an SMS containing a hyperlink, an HTML document containing a login form, a payment interface, an advertisement, etc.

[0028]

[0040] Exemplary events specific to mobile devices or particularly relevant to mobile device security include screen toggles (on / off), application label / name / icon changes, and screenshots. Other examples include requests to grant certain types of permissions (e.g., administrator, accessibility), permissions requested dynamically (i.e., at various stages of execution as opposed to installation time), and granting persistence (e.g., foreground services dynamically started by the respective application). Still other examples include attempts to prevent uninstallation of the respective application and displaying overlays on the OS settings interface (such overlays may trick unsuspecting users into granting unwanted permissions to the respective application).

[0029]

[0041] Event detection may be device-type specific and may include any method known in the art of computer security. In one example where client device 12 is a personal or laptop computer, upon detecting the launch of a target application 24, event harvester 32 registers the respective application and / or its associated process with the OS 22's event logging service (e.g., Event Tracking for Windows (ETW) or UNIX Syslog). In response, harvester 32 can receive notification of various events occurring during the execution of the respective processes, either in real time or in logged form. Event logging tools typically generate a list of event descriptors, including a timestamp for each event, a numeric code identifying the event type, an indication of the type of process or application that generated the respective event, and other event parameters. In such an embodiment, harvester 32 can detect the occurrence of a target event by parsing the respective event logs.

[0030]

[0042] In another example of event detection, the event harvester 32 can modify the set of native functions of the OS 22 by inserting redirection instructions (also known as hooks or patches). In this way, when a process running on the client device 12 calls the respective OS function, execution is redirected to a callback routine that notifies the harvester 32 about the attempt to execute the respective OS function. When the hooked function serves the monitored event (e.g., file creation, process launch, etc.), the attempt to call the respective function can serve as an indicator of the occurrence of the respective event.

[0031]

[0043] In yet another example of event detection, electronic communications sent by each client device may be detected by installing a proxy module configured to intercept Domain Name Service (DNS) queries and / or HTTP requests sent by each client device.

[0032]

[0044] Some operating systems, such as those running on smartphones and wearables, may not allow such operations. However, other tools may be available for detecting the occurrence of various events. For example, some OSs expose application programming interfaces (APIs) that allow for registering callbacks for different notifications, inspecting network traffic, SMS / MMS operations, detecting access to storage devices (e.g., SD cards), etc. Some embodiments of the event harvester 32 use the functionality of accessibility APIs to access on-screen content and detect user interactions with the respective device and / or application. Some embodiments further use artificial intelligence (e.g., natural language processing, computer vision, etc.) or other means of analyzing the content displayed on the screen.

[0033]

[0045] Event detection can have a substantial computational cost and therefore may impact the user experience, especially on mobile platforms, which generally have less computational power than personal computers. Furthermore, some event detection activities may require specific permissions and / or OS settings that may not be activated on all client devices 12a-d. To minimize the impact of event detection activities, some embodiments of the event harvester 32 are configured to allow some event detection activities to be selectively turned on or off. The decision to activate or deactivate detection of a particular event may be made locally or remotely, for example, by the security server 14. As shown in FIG. 2 , in some embodiments, the server 14 sends a data request 19 to the client 12, the data request 19 including an indication of the event category and an associated flag that instructs each event harvester 32 whether to monitor the respective category of event. In response to receiving the data request 19, the event harvester 32 can reconfigure the various event detection devices of each client to start and / or stop detecting the respective event type or category.

[0034]

[0046] In one exemplary use case scenario, to conserve resources and minimize impact on user experience, event harvester 32 may be configured to monitor each application installed on each client device for a predetermined period (e.g., minutes, hours, etc.) after its launch and then partially or entirely turn off the event detection activity of each application. Such an embodiment relies on the observation that common malware typically executes its payload early in the application's lifecycle. However, from time to time, if server 14 is notified of the occurrence of a suspicious event caused by a respective application on a protected client, server 14 can send data requests 19 to other clients running the respective applications, effectively requesting that their local event harvesters 32 resume event detection operations for the respective applications.

[0035]

[0047] In another exemplary use case scenario, the event harvester 32 may be configured to monitor only a subset of event types, which may be relatively inexpensive to detect and therefore have minimal impact on clients. However, if malware intelligence suggests that an attack is imminent, more computationally expensive specific event detection activities may be turned on from time to time. For example, the server 14 may select a set of clients according to their geographic location (e.g., clients in Scandinavia) and turn on detection of specific events on the selected clients based on knowledge of ongoing malware campaigns targeting clients located within the respective country / region.

[0036]

[0048] In yet another exemplary use case scenario, the security server 14 can divide the set of protected client devices 12a-d running a particular target application 24 into multiple subgroups, with each subgroup corresponding to a distinct subset of monitored events. In other words, the server 14 can instruct each subgroup of devices to listen only to its respective subset of event types, for example, by sending data requests 19 to each device, each request 19 specifying which subset of events to monitor on each device. Such a strategy can avoid expending valuable resources monitoring every event type (some of which may be particularly costly to detect) on every device. Meanwhile, the server 14 can reconstruct the “complete” behavior of each target application by merging event profiles collected from multiple devices, as described further below.

[0037]

[0049] In yet another exemplary use case scenario, the complete set of event types that can be monitored on a client device may be divided into (possibly overlapping) subgroups, with each event subgroup specific to a distinct category of malware and / or attack scenario. In other words, detection of an event from a selected subgroup may indicate the presence of a particular type of malicious agent or attack. In such an embodiment, event harvester 32 may be configured to monitor a relatively small subset of event types, including events selected from multiple event subgroups. In response to detecting the occurrence of an event (and thus suspected malware), harvester 32 may selectively turn on detection of other event types from the same subgroup as the triggering event.

[0038]

[0050] In some embodiments, the event harvester 32 is configured to attribute each detected event to the application (e.g., target app 24) running on the respective client causing the respective event to occur. In some embodiments, to accurately distinguish between various target applications, a unique application ID is calculated for each application. An exemplary application ID may be constructed according to the application name, and further according to a version indicator and an OS indicator (e.g., WhatsApp® version 2.22.17.70 for Android®). In another example, the application ID may include a hash of a portion of the respective application's code, such as an integrity hash used to verify the integrity of the application's code before installation or first execution on the respective device. In some embodiments, appending the respective application ID to the event indicator 16 enables the security server 14 to match behavioral data across multiple devices running the same instance of the respective target application. A verdict, such as “clean” or “malicious,” is then applied to all instances of the respective application, i.e., to all applications with the respective application ID, regardless of the client devices 12a-d on which they are running.

[0039]

[0051] In some embodiments, the security application 30 further comprises a device profiler module 34 configured to determine a device profile 17 and transmit the profile 17 to the security server 14. The profiler 34 can use any method known in the art to extract device profile data about each device, such as a device ID (e.g., an IMEI number), hardware type, hardware configuration, OS 22 type and version, various OS settings, current IP address, a list of currently installed applications, and current geographic location. Such information can be extracted, for example, via specific function calls to standard APIs exposed by the OS 22. Some device profile data, such as the current IP address, can be determined by the server 14 itself or by a proxy device (e.g., a network gateway device that mediates exchanges between each client device and the server 14), for example, according to communication metadata.

[0040]

[0052] FIG. 4 illustrates exemplary components of security server 14 according to some embodiments of the present invention. Behavior analyzer 36 may include a set of computer programs executing on the server 14's hardware processor and is configured to receive event indicators 16 and device profiles 17 from multiple client devices 12-d. Some of these event indicators 16 may describe events caused by the execution of multiple different instances of the same target application, each running on a different client device. Device profile 17, in turn, includes a token (e.g., a client ID) identifying each client device and other data characterizing the hardware and / or software of each device. In response to receiving device profile 17, some embodiments may create a client record corresponding to each client device and store each client record in security database 20.

[0041]

[0053] In some embodiments, behavior analyzer 36 is further configured to assemble an aggregate set and / or sequence of events that collectively describe the behavior of each target application across multiple client devices, as described in more detail below. Analyzer 36 can further analyze the aggregate set or sequence of events to derive a behavior signature 60 associated with each target application and determine whether each target application is malicious. Furthermore, behavior signature data can be stored in security database 20. The operation of behavior analysis is described in more detail below.

[0042]

[0054] In some embodiments, the security server 14 further includes a notification dispatcher 38 connected to the behavior analyzer 36, the notification dispatcher 38 configured to formulate and selectively transmit security notifications 18 and / or data requests 19 to the protected client devices 12a-d (FIG. 1). Transmitting in this context encompasses communicating directly with the respective clients and placing a computer-readable encoding of each message / notification in a repository (e.g., cloud storage) from which it may be selectively retrieved by the respective client devices.

[0043]

[0055] 5 shows an exemplary sequence of steps performed by security server 14 according to some embodiments of the present invention. Server 14 may be configured to continuously receive event indicators 16 and device profiles 17 from protected clients 12a-d (step 200) until an accumulation condition is met (step 204). Such accumulation effectively creates an event pool that will be analyzed by behavior analyzer 36 to detect malicious behavior.

[0044]

[0056] Exemplary accumulation conditions may include, among other things, comparing the count of events to a predetermined threshold for each target application 24 (as identified by a distinct application ID) and accumulating the data for a predetermined amount of time. More complex accumulation conditions may include criteria such as device type, geolocation, etc. For example, in step 204, the count of events detected on an Android smartphone from Germany may be compared to a predetermined threshold. Another exemplary embodiment may accumulate event indicators until it has a sufficient sample of events from various countries or regions.

[0045]

[0057] In some embodiments, the type and / or amount of forensic data collected by data requests 19 (step 202) targeted to particular client devices 12a-d may be further tailored. Such data requests may be used to selectively turn event detection on and / or off for particular event types, device types, device locations, etc., as described above.

[0046]

[0058] While events are accumulating (step 204 returns NO), individual event indicators 16 may be processed by a dedicated component of behavior analyzer 36, for example, by creating a database record for each event indicator, the record including at least an indicator of the respective event type, an indicator of the target application that caused the respective event (e.g., application ID), and an indicator of the client device on which the respective event occurred (e.g., client ID). Such records may be stored in security database 20 for future reference and / or behavior analysis, as described in more detail below.

[0047]

[0059] Once the event accumulation condition is met, step 206 can perform a behavioral analysis of the target application 24. In some embodiments, step 206 includes selecting events from the available event pool (e.g., the event indicators accumulated in steps 200-202-204) according to their cause, so that all selected events are caused by instances of application 24 (e.g., identified according to their respective application IDs). Step 206 further includes analyzing the selected set of events, as described in more detail below. While this specification focuses on only one target application, those skilled in the art will recognize that multiple target applications can be monitored simultaneously, for example, by similarly repeating step 206 for each application to be monitored.

[0048]

[0060] 6A-B illustrate an alternative exemplary sequence of steps performed by the server 14 (e.g., the behavior analyzer component 36, see FIG. 4) in step 206. In a simple embodiment, such as that shown in FIG. 6A, only a collection of individual events can be considered, without concern for the ordering of the respective events and / or the correlation between the individual events. In other words, in some embodiments, application behavior can be characterized by a simple set of events (e.g., requesting permission to access a user's address book and sending an HTTP request to a particular Internet domain) triggered by the respective application. To determine such a characteristic set of events, in step 302, the behavior analyzer 36 can select a set of client devices as event sources. In a simple example, the analyzer 36 can aggregate events from all available sources, i.e., all events triggered by each target application and reported during the current and / or previous accumulation cycles (steps 200-202-204 described above). In other embodiments, various device selection procedures may be applied to selectively aggregate events from, for example, selected device types (e.g., smartphones), devices running a particular type of OS 22 (e.g., Windows®, Android®, etc.), devices located in a selected region (e.g., Eastern Europe), etc. Such source selection may enable behavioral signatures to be determined with varying degrees of granularity and specificity, enabling sophisticated malware detection. Indeed, source selection may include identifying a subset of client devices according to device profiles 17 received from each client and selectively retrieving a set of event records from database 20 according to the client IDs of the selected client devices.

[0049]

[0061] In response to the selection of a set of client devices and / or events, in step 304, the behavior analyzer 36 may construct an aggregate event set that describes the behavior of each target application across the selected set of clients. As used herein, an aggregate event set represents a set constructed from events collected from multiple sources, i.e., a set that combines events occurring on one client device with events occurring on at least another client device. In an exemplary embodiment, the behavior analyzer 36 may construct the aggregate event set by union as follows:

[0050] A=∪ i∈Σ S i [1] where A denotes the aggregate event set, X denotes the selected set of client devices, and S i denotes the set of events caused by the execution of each target application and detected on client device i. In some embodiments, events received during the most recent accumulation cycle are combined with events previously detected on each client device, where all combined events were caused by an instance of the same target application. In some embodiments, constructing aggregate event set A further includes de-duplicating events, i.e., set A does not include two events of the same type.

[0051]

[0062] In some embodiments, in step 306 (FIG. 6-A), the behavior analyzer 36 can further determine whether the target application is malicious according to the contents of the aggregated event set A. Such a determination may be made according to any method known in the art of computer security. In one example, the behavior analyzer 38 can apply a set of previously derived heuristics to determine whether the event set A indicates maliciousness. For example, some events may indicate malware by themselves, while others may indicate maliciousness only when they occur in conjunction with other events. Such forensic knowledge may be derived automatically or by a security analyst based on the behavior of known malicious agents. Another exemplary embodiment can use artificial intelligence to determine whether the aggregated event set A indicates maliciousness. For example, a set of artificial neural networks may be pre-trained based on a training corpus of events characterizing the behavior of known malicious and benign applications. The trained network can then receive an encoding of the aggregated event set A and, in response, output a label (clean / malicious) or score indicating the likelihood that each target application is malicious. The structure and training of such artificial intelligence systems are beyond the scope of this specification, but several such examples are known in the art.

[0052]

[0063] In a further step 308, a behavioral signature of application 24 can be determined according to aggregate event set A. As shown in FIG. 6-A , in some embodiments, behavioral signature 60 includes a subset of aggregate event set A. Because set A was assembled from events recorded on multiple clients, distinct portions of signature 60 may have occurred on different client devices. To determine the behavioral signature, some embodiments of behavior analyzer 36 prune the aggregate event set, e.g., removing various events that are not considered characteristic of the behavior of the respective application and / or are not considered beneficial to security. In one such example, an event involving access to a popular web resource may not be included in the behavioral signature, but a request to grant a specific permission (e.g., access to the microphone) may be included. Some such criteria for extracting behavioral signatures from aggregate event sets and sequences are described further below.

[0053]

[0064] 6-B illustrates an exemplary sequence of steps performed by behavior analyzer 36 to determine a behavioral signature of target application 24 in a more sophisticated embodiment configured to analyze events in context, e.g., as an event sequence. In other words, in such an embodiment, the behavior of target application 24 is characterized not only by the set of individual events caused by the execution of each application, but also by the order in which each event occurred, the amount of time that elapsed between each event, and / or the correlations between other events.

[0054]

[0065] Step 322 may select a set of client devices, for example, according to the criteria described above in connection with step 302 of FIG. 6-A. Then, in step 324, for each selected device, the behavior analyzer 36 may construct a per-device event set by selecting events that occurred on the respective client device from the available collection of events. Some embodiments further arrange the per-device event set into an ordered sequence according to the time of occurrence of each event. In a further step 326, the per-device event sets and / or sequences may be merged into an aggregate event sequence that combines events that occurred on one device with events that occurred on at least another device of the selected set of clients. By combining events from multiple sources, the aggregate event sequence is said herein to describe the behavior of the target application across the selected client devices.

[0055]

[0066] The process of computing the aggregate event sequence is further illustrated in Figure 7-A-B-C, where the individual events are i , where i indexes the event type. In other words, two events with the same index are events of the same type. In the example shown in FIG. 7-A, two per-device event sets are composed of events detected and reported by client devices D1 and D2, respectively. Without loss of generality, devices D1 and D2 may represent any of client devices 12a-d in FIG. 1. The embodiment shown in FIG. 7-A further orders each event set according to the occurrence time of the members of the respective event set to form per-device event sequences 40a and 40b, respectively, and constructs an exemplary aggregate event sequence 50a by concatenating per-device sequences 40a-b.

[0056]

[0067] Another example of constructing aggregate event sequences is shown in FIG. 7-B, where exemplary per-device event sequences 40c and 40d are collected from devices D1 and D2, respectively. In constructing aggregate event sequence 50b from members of per-device sequences 40c-d, the illustrated embodiment orders events according to each event's application-specific occurrence time rather than the traditional "time of day." The exemplary application-specific time consists of the amount of time elapsed since the occurrence of a reference event on each device, where the reference event is caused by a local instance of the target application 24. For example, the application-specific time may include the time elapsed since the launch of the target application 24 on each device, the time elapsed since a user logged into the application 24 on each device, etc. Such an embodiment relies on the observation that events are generally not synchronized across multiple devices because different instances of an application 24 generally run independently of each other. However, by viewing events in "application time," patterns and anomalies may become more apparent because event histories across multiple devices are naturally aligned. For example, the fact that event E1 occurred at 2:34 AM on device D1 and at 8:15 PM on device D2 may be less informative in characterizing the behavior of target application 24 than the fact that event E1 occurred on both devices approximately 10 seconds after the launch of the respective applications. In FIG. 7B, per-device event sequences 40c-d are shown with application times t1 and t2 specific to devices D1 and D2, respectively. For example, the position of event E2 on the t1 line may be determined according to the amount of time separating the occurrence of each event from the launch of application 24 on device D1. Similarly, the position of event E5 on the t2 line may be determined according to the amount of time separating the occurrence of each event from the launch of application 24 on device D2. In constructing aggregate event sequence 50b, each individual event E in sequence 50b may be included in sequence 50b. iThe position of the event is the "application time" of each event, i.e., t1(E i ), and t2(E i ) can be determined according to

[0057]

[0068] In some embodiments, the aggregated event sequence is further processed. For example, the behavior analyzer 36 may determine whether an event deduplication condition is met, and if so, remove at least one instance of the duplicate event from the aggregated event sequence. An exemplary event deduplication condition may determine whether two events of the same type occurred on different clients at approximately the same application time. In FIG. 7-B, event E1 occurred approximately simultaneously on both devices, as did event E3, so the aggregated event sequence 50b retains only one instance of each. In some embodiments, the event deduplication condition may further include event metadata, particularly hardware type, OS version, geolocation, and event ubiquity (see the discussion of such metadata below in connection with FIG. 8). For example, some embodiments may dedupe only events that occur on a significant percentage of client devices, or only events that occur at approximately the same application time on devices with similar device profiles.

[0058]

[0069] FIG. 7-C illustrates yet another example of constructing an aggregate event sequence from events collected from devices D1 and D2. In the illustrated example, ordering individual events into per-device event sequences 40e-f reveals two alternative histories separating events E1 and E3. FIG. 7-C illustrates an exemplary aggregate event sequence 50c configured to represent such alternative behaviors. In such an embodiment, the behavior analyzer 36 may be configured to identify “anchor” events (such as E1 and E3 in the current example) within the set of events collected from selected devices and construct the aggregate event sequence 50c to allow multiple event histories only within the interval separating such anchor events. The analyzer 36 may identify anchor events using time-of-occurrence / simultaneity criteria, as described above in connection with FIG. 7-B. In some embodiments, some events may also be identified as anchors according to event type by determining that the respective events are particularly important in the lifecycle of the application 24. One example of such an “anchor” event involves granting a specific permission (e.g., access to the microphone), without which the application 24 may be prevented from performing certain activities.

[0059]

[0070] In some embodiments, sequence 50c can be encoded as a directed graph, or multiple subsequence "chunks" can be combined using Boolean operators, such as, for example, E1(E2OR(E4E5))E3E2. In yet another exemplary embodiment, aggregate event sequence 50c can include a list enumerating multiple alternative event sequences, each corresponding to a separate path through the directed graph. Some such alternative event sequences include events detected on one device and events detected on another device, such as the sequence E1E4E5E3E2 in the example of FIG. 7-C.

[0060]

[0071] Aggregate event sets and sequences may be enriched with various metadata to facilitate malware detection and / or behavioral signature determination. FIG. 8 illustrates an exemplary event sequence 50 and associated metadata in accordance with some embodiments of the present invention. For each event in the sequence, data characterizing the event sequence 50 may record an indicator of the event type and an indicator of the time of occurrence of the respective event. The occurrence time may include actual time of day, “application time,” or both. The example in FIG. 8 illustrates exemplary normalized application time spanning the interval between the application's launch (t=0) and termination (t=1). Real-time data may also be useful when malware communicates with a command and control (CNC) server that orchestrates malicious activity such as denial of service and other types of attacks. In such situations, the application's malicious payload is typically launched on all devices at approximately the same time. Therefore, data indicating that certain events occurred on multiple devices at approximately the same time may also be useful in detecting malicious intent, and in some embodiments, both actual time and application time are used in determining whether a target application 24 is malicious.

[0061]

[0072] Other exemplary event metadata may include a ubiquity metric indicating the number / percentage of devices that detected the respective event. In an exemplary embodiment, a high ubiquity metric indicates that the respective event is ubiquitous. The ubiquity metric may be scaled between predetermined boundaries (e.g., between 0 and 1, with 1 indicating that the respective event was detected on all devices in the respective sample). Still other exemplary event metadata may include a location indicator indicating the location of client devices that detected the respective event. In the illustrated example, the location indicator includes a list of country codes (e.g., two-letter country codes defined by the ISO 3166 standard). Including geolocation information may be useful for characterizing applications that behave differently in different regions. Alternative location metadata may include a network address (e.g., IP address), which may be useful for detecting malicious software targeting specific businesses or public places such as malls, airports, schools, etc.

[0062]

[0073] Further example metadata that may annotate events in aggregate event set / sequence 50 include device profile data (e.g., OS version, hardware type, installed apps, etc.) and / or user profile data, as determined, for example, by profiler 34 as described above. Such profile data may be useful in determining whether target application 24 is malicious and / or in understanding previously unseen malware techniques. For example, malware often exploits vulnerabilities specific to particular device types and OS versions. Some malware targets specific assets, such as a particular bank's internet banking application or a particular trading platform. Some malware (e.g., Flubot) only attacks devices with specific applications or categories of applications (e.g., cryptocurrency apps) installed. Some malware (e.g., Blackrock) propagates within social networks via phishing hyperlinks distributed via Instagram®, Facebook®, WhatsApp®, etc. Thus, the relevance of a particular event may vary depending on whether the respective client device has installed such applications. In yet another example, some malware may use Quick Response (QR) codes to direct users to malicious internet content. However, such malware may only propagate on devices running specific applications known to display QR codes, for example, as online advertisements. User profile data can also be useful in forensic analysis. For example, some malware may specifically target users with specific interests (e.g., gaming, entertainment, shopping) revealed by their online activity and installed apps.

[0063]

[0074] In some embodiments, such as shown in FIG. 6-B , the aggregate event sequence can then be used in step 328 to determine whether application 24 is malicious. Step 328 may use any method or algorithm known in the art of computer security, such as statistical data or a set of heuristics / rules derived from previously observed malware infections. In one exemplary approach, a reference probability for each event or a length of event sequence is determined by analyzing a corpus of events caused by known malicious and clean applications. The reference probability may indicate, for example, the probability that a respective sequence of events belongs to a malicious application. In step 328, behavior analyzer 36 can decompose the aggregate event sequence determined in step 326 into subsequences with known reference probabilities. In some embodiments, the likelihood that the aggregate event sequence is malicious is then calculated by combining the respective reference probabilities.

[0064]

[0075] In determining whether an aggregate event sequence is indicative of maliciousness, some embodiments further consider the time gap between events, either in real time or "application time." Some event sequences are indicative of malware if the events occur in succession (bursts) as opposed to being separated by relatively large gaps. Such bursts may be detected in application time. In contrast, some event types may be indicative of malware if they are concentrated in real time. Such a concentration of events in real time may indicate a coordinated attack (e.g., denial of service) orchestrated remotely by a malicious command and control server.

[0065]

[0076] Another exemplary method uses a pre-trained artificial intelligence system configured to ingest aggregate event sequences and, in response, generate a label or score indicating whether each event sequence is malicious. Such methods and systems are beyond the scope of this specification. Suffice it to say that several types of neural networks are known that explicitly process sets of inputs according to the ordering of each input. Examples include various convolutional neural network (CNN) and recurrent neural network (RNN) architectures.

[0066]

[0077] In a further step 330 (FIG. 6-B), the behavior analyzer 36 can determine a behavioral signature of the application 24 according to the aggregate event sequence. In some embodiments, the behavioral signature 60 includes a subset of characteristic events extracted from the aggregate event sequence, with each characteristic event ordered according to the respective event's time of occurrence. For example, the signature 60 can include a subsequence of the aggregate event sequence determined in step 326. Because the aggregate event sequence was assembled from events recorded at multiple clients, different portions of the behavioral signature 60 can have occurred on different client devices.

[0067]

[0078] In some embodiments, behavioral signature 60 characterizes the behavior of target application 24 as a whole, not just its malicious aspects. For example, in the case of a malicious application masquerading as legitimate / legitimate, signature 60 may include the benign actions of target application 24. In other embodiments, behavioral signature 60 may be constructed to selectively capture only a subset of events / actions that individually or collectively indicate malicious behavior. Such behavioral signatures may further be used to detect malicious intent in other, previously untested applications.

[0068]

[0079] To construct the behavioral signature 60, the behavior analyzer 36 may analyze the aggregate event set / sequence and determine whether to include each element of the aggregate set / sequence in the signature 60 according to statistical, forensic, and / or other criteria. One exemplary statistical criterion includes the frequency and / or ubiquity of the selected event. Events occurring on many client devices (e.g., a high ubiquity measure) may be considered characteristic of the behavior of the respective application and therefore may be included in the signature 60. However, each event type may not be particularly informative or indicative of malware and thus may be omitted from behavioral signatures explicitly configured for malware detection. For example, in the situation shown in FIGS. 7A-7C, event type E3 is ubiquitous but is not included in the signature 60. Certain events may not indicate malware when occurring in isolation, but may indicate malicious intent when occurring together with other types of events or in certain sequences. For example, the sequence E4E5 in Figures 7A-B-C was included in signature 60 because it may be known to be associated with a particular malicious agent and / or attack strategy. Another exemplary event type typically included in signature 60 is a gatekeeper event that conditions the occurrence of other events. One such exemplary gatekeeper event includes setting specific permissions on a mobile device. For example, if the malicious payload of application 24 includes tracking each client device, such malicious activity cannot be carried out without first granting application 24 access to geolocation data.

[0069]

[0080] In some embodiments, in response to determining the behavioral signature 60, the security server 14 may store an encoding of the signature 60 in the security database 20 in step 208 (FIG. 5). The signature 60 may be further incorporated into computer security software and / or protocols used to detect malicious software. If the behavioral analysis determines that the application 24 is not malicious (e.g., step 210 returns no), some embodiments return to accumulating event indicators 17 and device profiles 16 from the protected client devices 12a-d (FIG. 1). If a determination is made that the target application 24 is malicious, the sequence of steps 212-214 may select a set of client devices to notify and notify the respective devices. Some security notifications 18 may explicitly warn the respective device users of a possible malware infection and provide mitigation instructions. In some embodiments, all client devices 12a-d on which the respective target application is installed and / or running are notified. Other exemplary embodiments may select a subset of the respective devices according to various criteria, such as OS version or geographic location. In one example, forensic analysis according to the aggregate event set / sequence described above may reveal that the target application 24 exploits a particular vulnerability associated with a particular version of the OS 22. In response, in step 212, the notification dispatcher 38 may select client devices 12a-d currently running the respective OS versions. In another example, the behavior analyzer 36 may determine that the target application 24 specifically targets client devices in Germany and Scandinavian countries. In response, step 212 may include selecting for notification only client devices currently located in the respective regions. Yet another exemplary embodiment may select clients to notify according to IP address, etc.Step 212 may rely on device profile data 17 collected from each device and / or stored in security database 20 .

[0070]

[0081] While the above description illustrates various methods and algorithms that may be embodied as computer programs executed by a general-purpose hardware processor, those skilled in the art will appreciate that each function may also be implemented using specialized hardware components, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). FIG. 9 illustrates an exemplary hardware configuration of a computer system 80 programmable to execute some of the methods and algorithms described herein. The illustrated configuration is general and may represent any of the client devices 12a-d and / or the security server 14. Those skilled in the art will appreciate that the hardware configurations of some types of devices (e.g., mobile phones, smartwatches, servers, routers) may differ somewhat from that illustrated in FIG. 9.

[0071]

[0082] The illustrated computer system comprises a set of physical devices including a hardware processor 82 and a memory unit 84. The processor 82 comprises a physical device (e.g., a microprocessor, a multi-core integrated circuit formed on a semiconductor substrate, etc.) configured to perform computational and / or logical operations on sets of signals and / or data. In some embodiments, such operations are specified to the processor 82 in the form of a sequence of processor instructions (e.g., machine code) that encode algorithms for performing some of the methods described herein. The memory unit 84 may include a volatile computer-readable medium (e.g., DRAM, SRAM) that stores instructions and / or data accessed or generated by the processor 82.

[0072]

[0083] Input devices 86 may include, among other things, a computer keyboard, mouse, and microphone, including respective hardware interfaces and / or adapters that enable a user to introduce data and / or instructions into the respective computer systems. Output devices 88 may include, among other things, display devices such as monitors and speakers, and hardware interfaces / adapters such as graphics cards, that enable the illustrated computing equipment to communicate data to a user. In some embodiments, input devices 86 and output devices 88 share common hardware, as in the case of a touchscreen device. Storage devices 92 include computer-readable media that enable non-volatile storage, reading, and writing of software instructions and / or data. Exemplary storage devices 92 include magnetic disks, optical disks, and flash memory devices, as well as removable media such as CD and / or DVD disks and drives. A set of network adapters 94, along with associated communication interfaces, enable the illustrated computer system to connect to communications network 15 (FIG. 1) and / or other devices / computer systems. Controller hub 90 generally represents multiple system, peripheral, and / or chipset buses and / or all other circuitry that enables communication between processor 82 and devices 84, 86, 88, 92, and 94. For example, controller hub 90 may include, among other things, a memory controller, an input / output (I / O) controller, and an interrupt controller. In another example, controller hub 90 may comprise a northbridge that connects processor 82 to memory 84 and / or a southbridge that connects processor 82 to devices 86, 88, 92, and 94.

[0073]

[0084] The exemplary systems and methods described above enable protection of electronic devices and their users from sophisticated malicious software. Some of the described methods and systems are particularly suited to mobile computing devices, such as smartphones, tablet computers, wearable computers, etc. Such mobile devices typically have significantly less computing power, memory, and storage than other computer systems, such as personal computers or servers, and therefore can pose unique challenges to traditional computer security paradigms.

[0074]

[0085] Conventional anti-malware software typically monitors software behavior by detecting the occurrence of selected events and applying a set of rules and / or calculations to the detected events to determine whether a respective device is infected. However, observing application behavior can be technically challenging, especially on mobile computing devices. Security software consumes significant resources, which can negatively impact the user experience. Some event detection methods, such as hooking various OS functions, are not available on all operating systems and device types. The same application may behave differently on different devices because some aspects / functions of each application may require specific OS settings (e.g., certain permissions that are not granted on all devices).

[0075]

[0086] Furthermore, some sophisticated malware actively attempts to evade detection by security software. Some malware agents carefully select their victims by assessing the value or vulnerability of users or the potential reward for attacking a particular host. In other words, an agent may decide to deploy its malicious payload to only a relatively small percentage of hosts it infects, while behaving as benign, legitimate software disguised as, for example, a utility or entertainment application to all other hosts. Other malicious agents are active only on specific types of hardware and / or operating system versions. Still other malicious agents selectively target clients in specific regions, countries, carriers, etc. In yet another example, a malicious agent may refrain from deploying its malicious payload for a relatively long time interval, thereby fooling security software into classifying it as benign. Some such agents are remotely activated by a signal received from a remote entity (commonly known as a command and control server), which may arrive days or even months after the respective application is installed on the respective device.

[0076]

[0087] The device characteristics and malware evasion strategies described above complicate anti-malware activities for at least the following reasons: First, the same application may behave differently on different devices, making it difficult to observe and characterize the behavior of malicious applications "in the wild." Second, because some applications only expose their malicious aspects to a relatively small percentage of infected hosts, security software may erroneously determine that a client is clean when, in fact, the client is infected with a malware agent that is not currently active for various reasons.

[0077]

[0088] Some embodiments of the present invention explicitly address such shortcomings, thereby enhancing the security of mobile computing devices. In some embodiments, an event harvester runs on each protected device and is configured to detect the occurrence of various events caused by the execution of a target application (e.g., a mobile app) on the respective device. However, in some embodiments, instead of separately analyzing each locally detected event set, the respective events are reported to a central security server, which then collates the behavior of each target application across multiple devices. Some embodiments calculate an aggregate event set and / or sequence that combines events detected on one device with events detected on other devices. The security server then determines whether the target application is malicious according to the aggregate event set / sequence. In response to a malicious determination, some embodiments send a security notification to devices running the respective target applications, thereby enabling users of the respective devices to remove the problematic software or otherwise mitigate its effects. Furthermore, some embodiments can extract a behavioral signature that includes a reduced subset / subsequence of the aggregate event set / sequence, for example, including events or actions essential to describing the malicious modus operandi of the respective application.

[0078]

[0089] Intuitively, an aggregate event set / sequence describes the behavior of a "virtual application" that collectively represents the set of all individual instances of each target application running on the devices that provided the content of the aggregate set / sequence. When different instances behave differently on different devices for any of the reasons outlined above, the aggregate event set / sequence captures a unified view of all such behavioral variations.

[0079]

[0090] In some embodiments, a single aggregate event sequence is constructed by ordering events received from multiple devices according to a device-specific "application time," which consists of the time elapsed between the occurrence of each event and a local reference, such as the moment each instance of the target application was installed or launched on each device. In some embodiments, events that occur identically on multiple devices are then de-duplicated. Using application time instead of actual time in constructing the aggregate event sequence can facilitate security analysis by aligning individual event timelines across multiple devices with each other to reveal behavioral patterns and / or anomalies.

[0080]

[0091] Aggregating events from multiple devices as demonstrated herein enables rapid response to emerging security threats. By simultaneously listening to multiple instances of the same application, a security server can assemble related characteristic event sets / sequences and detect malicious content in significantly less time than required by traditional methods that rely on per-device event sets / sequences. Several monitored instances of a target application may currently be at different stages of their lifecycles, allowing for highly efficient data collection. Furthermore, some embodiments rely on the observation that if an application's malicious payload is activated only on a relatively small percentage of infected devices, simultaneously monitoring many instances of each application increases the chance of seeing indicators of malicious behavior.

[0081]

[0092] Assembling events from multiple sources further provides rich forensic data that captures behavioral variations (e.g., polymorphism), thereby enabling the extraction of uniform behavioral signatures that can be used to effectively detect malware on a variety of devices. The sample of devices providing behavioral data to the server may be purposefully tailored according to various criteria, such as to maximize diversity or to emphasize certain geographic regions over others. Such sophisticated data collection can enable the extraction of behavioral signatures with various granularities.

[0082]

[0093] A strategy of aggregating events across multiple devices can further facilitate malware detection on mobile devices by reducing the computational load associated with security activities. In some embodiments of the present invention, instead of listening to a comprehensive set of event types on each device, behavior monitoring tasks can be divided among multiple devices, for example, by selectively turning on or off some event detectors to conserve computational resources. A security server can orchestrate such selective data collection across multiple devices by sending data requests to each client device, where the data requests specify a subset of event types to listen for on each device. In such embodiments, some potentially relevant events may not be detected on some devices but may be recorded on other devices and thus contribute to the aggregated event set / sequence collected at the security server.

[0083]

[0094] It will be apparent to those skilled in the art that the above-described embodiments can be modified in many ways without departing from the scope of the present invention. Therefore, the scope of the present invention should be determined by the appended claims and their legal equivalents.

Claims

1. 1. A method for protecting a plurality of client devices from malicious software, comprising: using at least one hardware processor of a computer system, selecting a plurality of events from the event pool according to causes of the plurality of events, the plurality of selected events all being caused by the target software application; a first event of the plurality of events caused by an instance of the target application executing on a client device of the plurality of client devices; a second event of the plurality of events caused by another instance of the target application running on another client device of the plurality of client devices; Steps and ordering the selected plurality of events according to a time of occurrence of each event to form an aggregate event sequence including the first event and the second event; determining whether the target application is malicious according to the aggregate event sequence; A method comprising:

2. 2. The method of claim 1, wherein arranging the selected events includes determining a position of the first event within the aggregate event sequence according to a length of time separating an occurrence of the first event from a launch of the one instance of the target application on the client device.

3. 2. The method of claim 1, wherein the step of ordering the selected events comprises: determining whether an event deduplication condition is met according to an occurrence time of the first event and further according to an occurrence time of the second event; in response, removing duplicate instances of the first event or the second event from the aggregated event sequence when the event de-duplication condition is met; A method comprising:

4. 2. The method of claim 1, further comprising determining whether the event deduplication condition is satisfied according to a time of launch of the one instance of the target application on the client device and according to a time of launch of the other instance of the target application on the other client device.

5. 2. The method of claim 1, wherein the step of sequencing the selected plurality of events comprises concatenating a first sequence of events with a second sequence of events; the first event sequence includes a first subset of the plurality of events, the first subset being selected to include only events caused by the one instance of the target application and ordered according to the time of occurrence of each member of the first subset; the second event sequence includes a second subset of the plurality of events, the second subset being selected to include only events caused by the other instance of the target application, and being ordered according to the time of occurrence of each member of the second subset.

6. 2. The method of claim 1, further comprising determining whether the target application is malicious according to a number of the plurality of client devices that report the occurrence of an event of the same event type as the first event, the event being caused by a local instance of the target application.

7. The method of claim 1 , comprising determining whether the target application is malicious further according to a geographic location of the client device.

8. 2. The method of claim 1, comprising determining whether the target application is malicious according to a time of occurrence of the first event and further according to a length of time separating the occurrence of the first event from the launch of the one instance of the target application on the client device.

9. 2. The method of claim 1, further comprising determining whether the target application is malicious according to a length of time separating the occurrence of the first event on the client device from the occurrence of the second event on the other client device.

10. 2. The method of claim 1, wherein the client device comprises an event harvester configured to detect events occurring at the client device for inclusion in the event pool, the method further comprising using the at least one hardware processor to send a data request to the client device in preparation for selecting the plurality of events, the data request being formulated to cause the event harvester to suspend detection of events of an event type of the second event.

11. 1. A computer system having at least one hardware processor, the at least one hardware processor comprising: selecting a plurality of events from the event pool according to causes of the plurality of events, the plurality of selected events all being caused by the target software application; a first event of the plurality of events caused by an instance of the target application executing on a client device of a plurality of client devices; a second event of the plurality of events caused by another instance of the target application running on another client device of the plurality of client devices; To choose and arranging the selected plurality of events according to a time of occurrence of each event to form an aggregate event sequence including the first event and the second event; determining whether the target application is malicious according to the aggregate event sequence; A computer system configured to:

12. 12. The computer system of claim 11, wherein arranging the selected events includes determining a position of the first event within the aggregate event sequence according to a length of time separating an occurrence of the first event from a launch of the one instance of the target application on the client device.

13. 12. The computer system of claim 11, wherein arranging the selected events comprises: determining whether an event deduplication condition is satisfied according to an occurrence time of the first event and further according to an occurrence time of the second event; in response, removing duplicate instances of the first event or the second event from the aggregated event sequence when the event de-duplication condition is satisfied; 2. A computer system comprising:

14. 14. The computer system of claim 13, wherein the at least one hardware processor is configured to determine whether the event deduplication condition is satisfied further according to a time of launch of the one instance of the target application on the client device and further according to a time of launch of the other instance of the target application on the other client device.

15. 12. The computer system of claim 11, wherein sequencing the selected plurality of events includes concatenating a first sequence of events with a second sequence of events; the first event sequence includes a first subset of the plurality of events, the first subset being selected to include only events caused by the one instance of the target application and ordered according to the time of occurrence of each member of the first subset; the second event sequence includes a second subset of the plurality of events, the second subset being selected to include only events caused by the other instance of the target application, and ordered according to the time of occurrence of each member of the second subset; Computer system.

16. 12. The computer system of claim 11, wherein the at least one hardware processor is configured to determine whether the target application is malicious further according to a number of devices among the plurality of client devices that report the occurrence of an event of the same event type as the first event, the event being caused by a local instance of the target application.

17. 12. The computer system of claim 11, wherein the at least one hardware processor is configured to determine whether the target application is malicious further according to a geographic location of the client device.

18. 12. The computer system of claim 11, wherein the at least one hardware processor is configured to determine whether the target application is malicious according to a time of occurrence of the first event and further according to a length of time separating the occurrence of the first event from a launch of the one instance of the target application on the client device.

19. 12. The computer system of claim 11, wherein the at least one hardware processor is configured to determine whether the target application is malicious further according to an amount of time separating an occurrence of the first event on the client device from an occurrence of the second event on the other client device.

20. 12. The computer system of claim 11, wherein the client device comprises an event harvester configured to detect events occurring at the client device for inclusion in the event pool, and wherein the at least one hardware processor is further configured to send a data request to the client device in preparation for selecting the plurality of events, the data request being formulated to cause the event harvester to suspend detection of events of an event type of the second event.

21. A non-transitory computer-readable medium having stored thereon instructions that, when executed by at least one hardware processor of a computer system, cause the computer system to: selecting a plurality of events from the event pool according to causes of the plurality of events, the plurality of selected events all being caused by the target software application; a first event of the plurality of events caused by an instance of the target application executing on a client device of a plurality of client devices; a second event of the plurality of events caused by another instance of the target application running on another client device of the plurality of client devices; To choose and arranging the selected plurality of events according to a time of occurrence of each event to form an aggregate event sequence including the first event and the second event; determining whether the target application is malicious according to the aggregate event sequence; A non-transitory computer-readable medium for causing

Images