Malware undetectable sandbox

Abstract

Embodiments are directed to preventing sandbox environments from being detected by potentially malicious applications. To this end, execution of an application is monitored and information about the execution is provided to a reinforcement learning machine learning model. The model generates suggested modifications to be made to the executing application. The model is provided information indicating whether the application successfully executed and this information is used to train the model for other modifications. By modifying the execution of potentially malicious software during execution, detection of the sandbox environment can be prevented and better analysis of potentially malicious application features can be understood.

Description

Background Technology

[0001] Dynamic malware analysis observes the behavior of suspicious code when each sample of potential malware is detonated (executed) in a controlled analysis environment (sandbox). The actions of the suspicious code are then studied. The results of these studies may include Indicators of Compromise (IOCs). Furthermore, the behavior of the suspicious code during execution is used to determine whether the code is malicious. The data extracted from the IOCs will be used for:

[0002] • Expand defenses against future variants,

[0003] • Prevent other related malicious activities.

[0004] • Analyze malware families of interest to understand their operation.

[0005] • Combat responsible malware actors

[0006] Many sandboxes are implemented as virtualized systems that can be easily rolled back to a clean state once analysis is complete. However, these virtualized systems may not be able to truly simulate a real user's machine in one or more ways. Given the use of sandboxes to identify malware threats, attackers have employed various techniques to detect the presence of such virtual environments and, based on this, to detect actions that alter the behavior of malware applications. Furthermore, some malware today is designed to use multiple different evasion techniques to bypass detonation. Significant time and effort are spent by malware researchers identifying these evasion techniques and patching detonation environments to evade such checks. Summary of the Invention

[0007] This disclosure provides a scheme for detonating a potentially malicious application in a sandbox environment. In this scheme, while the potentially malicious application is executing in the sandbox environment, the sequence of instructions to be executed by the application is intercepted, and information about the execution is provided to a reinforcement learning machine learning model. The model can generate suggested modifications to the executing application. The model can also be provided with information indicating whether the application has successfully executed, and this information is used to train models for other modifications. This disclosure, by modifying the execution of the potentially malicious application during execution, can prevent detection in the sandbox environment and provide a better understanding of the characteristics of the potentially malicious application. Attached Figure Description

[0008] Figure 1 This is an overview diagram of an example system implementing one or more of the disclosed embodiments.

[0009] Figure 2 This is an overview diagram of an example system implementing one or more of the disclosed embodiments.

[0010] Figure 3 This is a data flow diagram of an example data flow within an example sandbox architecture implemented in one or more of the disclosed embodiments.

[0011] Figure 4 Another embodiment of a sandbox architecture implemented by one or more of the disclosed embodiments is shown.

[0012] Figure 5 Example machine learning modules are shown according to some examples of this disclosure.

[0013] Figure 6 This is a flowchart of the method used to modify the execution of the application.

[0014] Figure 7 This is a flowchart of the method used to modify the execution of the application.

[0015] Figure 8 A block diagram of an example machine 800 is shown, in which any one or more of the techniques (e.g., methods) discussed herein can be performed. Detailed Implementation

[0016] The disclosed embodiments provide improved methods and systems for malware detonation. As mentioned above, many security researchers rely on sandbox environments to analyze malware applications and better understand their behavior. This understanding is used for a variety of purposes, including developing more effective malware detection techniques, identifying security vulnerabilities in existing software, and providing signature or behavioral profiles to improve runtime detection of malware applications.

[0017] As malware becomes increasingly sophisticated, techniques have been introduced by malware authors to detect when malware is running in a sandbox environment. Once this is detected, the malware suppresses one or more of its functions, making forensic collection more difficult and / or impossible. For example, some malware monitors the number of CPUs on the system executing the malware. Because many sandbox environments are virtualized, the number of CPUs reported by the operating system is often small, such as one (1). In contrast, many common desktop computers contain multiple CPUs and typically report more numbers. In some embodiments, malware uses a small number of CPUs as evidence that it may be running in a sandbox environment. Another technique is to analyze the "Documents" folder for malware. Some malware will take the number of files contained in the document date and / or the modification date of the document as evidence of a sandbox environment if the document date is not typical for a real user's computer. Thus, a technical problem arises that it may become increasingly difficult to detonate malware in a sandbox testing environment, where information about the malware can be obtained through analysis and mitigation measures taken based on the analysis.

[0018] To address this technical challenge and make the sandbox environment less visible to potential malware being analyzed, the disclosed embodiments intercept malware program execution to mask certain characteristics of the sandbox environment and ensure that malware execution behaves consistent with analysis in a real, non-sandbox environment. If the malware cannot detect the sandbox, it will generally function normally, allowing for a comprehensive analysis of its operational methods.

[0019] To prevent detection of the sandbox environment, some embodiments intercept or otherwise hook the programmatic execution of suspicious malware applications. In some embodiments, each individual instruction, such as assembly-level or machine-level instructions, bytecode, pre-compiled code (p-code), other intermediate code instructions, and even source-level instructions in some embodiments, is intercepted and fed to a reinforcement learning model. The reinforcement learning model is trained to provide modification instructions based on the intercepted assembly instructions. The modification instructions indicate one or more modifications to the program execution of the suspicious malware application. For example, in various embodiments, the modification instructions indicate one or more of the following: modifying register values, modifying function return values, modifying function input parameters, modifying conditional evaluations in the malware code, or modifying branch instructions in the malicious code.

[0020] Some embodiments do not intercept instructions executed by the aforementioned suspected malware, but instead intercept function calls to one or more external application programming interfaces (APIs). For example, some embodiments intercept JavaScript library API calls and / or operating system API calls. When intercepting API calls, information related to the API call is provided to the reinforcement learning model. This information may include input parameters passed to the API call, the call stack of the suspected malware application, and, in some cases, indications of code from the suspected application adjacent to the API call. As described above, the reinforcement learning model is configured to execute modification instructions based on the provided information provider. In various embodiments, the reinforcement learning model instructs modifications to the API's return value, API's output parameters, API's input parameters, conditional evaluations within the suspected malware application itself, branch instructions within the suspected malicious software application itself, or other modifications.

[0021] Figure 1This is an overview diagram of an example system 100 implementing one or more of the disclosed embodiments. System 100 includes a web crawler computer 102 that accesses network 104 (e.g., the Internet) via firewall 106. The web crawler computer 102 identifies available data via network 104 (e.g., through websites) and identifies applications that attempt to run on the computer downloading the data. For example, in some examples, the web crawler identifies websites that attempt to download and install executable files on the downloading machine, and / or run scripting language applications in a browser environment provided by the downloading computer. In some embodiments, these identified applications are provided by the web crawler computer 102 to a sandbox environment 108. In some embodiments, potentially malicious applications are manually identified. The sandbox environment 108 attempts to execute these applications. The sandbox environment 108 is at least partially isolated from other environments within system 100 to mitigate any potential harm that may result from the execution of these identified applications.

[0022] One of the results of the analysis performed in the sandbox environment 108 is the identification of malware applications, or applications with malicious intent when executed on a specific computer. Malware applications can have a variety of objectives, including data destruction, data collection, or continuous surveillance of networks they can access. Some malware applications use their hosts as proxies for other malicious activities. For example, some denial-of-service attacks are orchestrated by a large number of other innocent computers infected with malware automatons configured to carry out denial-of-service attacks upon receiving commands from a central controller; in some cases, these computers are located in overseas jurisdictions that are difficult to manage.

[0023] Once a malicious application is identified, in some embodiments, it is stored in a malicious software data store 110. The malicious software data store 110 is then used in various embodiments to improve protection against the malicious software application. For example, in some embodiments, the malicious software stored in the malicious software data store 110 is studied to identify behavioral patterns detectable by dynamic threat assessment applications that protect most modern computing systems. In at least some embodiments, the malicious software stored in the malicious software data store 110 is also used to identify vulnerabilities in the design and / or architecture of widely used computer operating systems and / or applications.

[0024] Figure 2This is an overview diagram of an example system 200 performing one or more of the disclosed embodiments. System 200 includes a browser application 202 and a first potentially malicious application 204. Each of the browser application 202 and the first potentially malicious application is an application managed by an operating system 206. Both the browser application 202 and the first potentially malicious application 204 interface with the operating system 206 via an operating system application programming interface (API) 208. When the operating system 206 is a Microsoft Windows-based operating system, in at least some embodiments, the browser application 202 and the first potentially malicious application 204 are "exe" or ".com" files. When the operating system 206 is a Linux operating system, in some embodiments, the browser application 202 and the first potentially malicious application 204 contain target code compatible with the hardware platform running the operating system 206.

[0025] Figure 2 A second potential malicious software application 210 is also shown. The second potential malicious software application 210 differs from the first potential malicious software application 204 because it runs within an environment provided by browser application 202. In some embodiments, the second potential malicious software application is a JavaScript application. Browser application 202 provides a script API 212 for use by the second potential malicious software application 210 running within the environment provided by browser application 202. The second potential malicious software application 210 interfaces with script API 212 to perform various functions. In some embodiments, script API 212 calls OS API 208 as needed to perform at least some of these functions.

[0026] As described below, the disclosed embodiments intercept instructions and / or API calls executed by a first potential malware application 204 and / or a second potential malware application 210. These intercepted instructions and / or API calls are provided to a machine learning model, and modifications are to be made to the first potential malware application 204 and / or the second potential malware application 210.

[0027] Figure 3 This is a data flow diagram of an example data flow within an example sandbox architecture 300 implemented in one or more of the disclosed embodiments. Figure 3A potential malicious application 302 is illustrated. We refer to this application as potential malicious application software 302 because in some cases it is a malicious application, while in others it may be a completely benign application. Potential malicious application 302 contains code 304. In some embodiments, code 304 is executable code. For example, in some embodiments, code 304 is native assembly code or machine language code compatible with the hardware platform and operating system on which the potential malicious application executes. In other embodiments, code 304 is intermediate code, such as p-code or even script source code, which is at least partially interpreted before execution by the hardware processor.

[0028] Figure 3 An API hook 306 implemented by some disclosed embodiments is shown. API hook 306 is configured to intercept function calls to one or more APIs (e.g., script API 212 or OS API 208) executed by a potential malicious application 302 and code 304. In some embodiments, API hook 306 is a debugging application. In these embodiments, the debugging application is configured to intercept program execution of the potential malicious application 302 when it accesses an address outside of the potential malicious application 302's code 304 (e.g., accessing an API library, such as API library 309 discussed further below). In some embodiments, the operating system supporting the sandbox architecture 300 (e.g., operating system 206) provides API hooking capabilities, thus eliminating the need for a debugger. Some embodiments utilize a hardware emulator to implement API hook 306. For example, in some embodiments, the computer's traditional hardware processor is replaced by a hardware emulator capable of simulating the operation of OEM hardware circuitry and allowing monitoring and / or interception of programming activities performed by the emulated hardware processor.

[0029] Upon intercepting API call 308 (or any function call), API hook 306 determines information associated with the hooked API call. In various embodiments, this information includes one or more indicators of the function name 310 of API call 308, the value of the input parameter 312 passed from the potential malware application 302 to API call 308, the call stack of the potential malware application 302 when API call 308 was made, the code of the potential malware application 302 near or adjacent to API call 308, and other information from potential malware application attributes 314 (e.g., the name of the potential malware, the size of the potential malware, etc.) or machine learning model 316. Regarding embodiments providing code adjacent to the API call, some embodiments provide a predefined number of bytes of code in the image (file) of the potential malware application 302 prior to the API call, and a second predefined number of bytes of code in the image (file) of the potential malware application 302 after the API call. Some embodiments provide a predefined number of instructions (assembly / machine instructions, intermediate code instructions, or source instructions) of the potential malware application before and after the API call.

[0030] Model 410 contains data defining the relationship between previously suggested modifications and the successful or unsuccessful execution of a potential malicious application, as described below, as well as an algorithm for determining modifications based on inputs provided to model 410, and the data defining the relationships. Based on information provided by API hook 306, machine learning model 316 is trained to determine suggested modifications 318 for the potential malicious application 302. In some embodiments, the ML model implements a reinforcement learning algorithm. In some embodiments, machine learning model 316 is trained to generate suggested modifications that maximize the probability of successful completion of the potential malicious application 302. Successful completion has various definitions depending on the specific embodiment, but sets one or more conditions for the execution of the potential malicious application 302. Some embodiments define successful completion as the potential malicious application establishing a network connection with a remote device, creating and / or writing files, writing to the system registry (e.g., the Microsoft Windows Registry), or generating one or more new processes or threads (different from the malicious application itself). Some embodiments evaluate whether the potential malicious application 302 creates or writes any files as a criterion for evaluating whether the malicious application operates successfully. Some embodiments count the number of API calls issued by the potential malicious application. Some implementations determine whether a potential malicious software application has been successfully executed based on whether the count exceeds a predetermined threshold.

[0031] The proposed modification 318 is provided to the execution modifier component 320, which implements modification 322 when executing the potential malware application 302. In some embodiments, the proposed modification 318 indicates a modification to the API return value of API call 308. In this case, the execution modifier invokes the real API corresponding to API call 308 from API library 309, but replaces any return value from the API in API library 309 with a different return value. In some embodiments, the different return value is selected from a public (e.g., N most frequently returned) list of return values ​​of the hooked API. The selected return value is then used as the different return value. In some embodiments, the proposed modification is not a modification. In this case, the execution modifier 320 merely acts as a proxy and invokes the hooked API from the real API library, passing any input parameters, output parameters, and return values ​​in an unmodified form.

[0032] In some embodiments, the proposed modification is a modification of the execution of a portion of code contained within the potential malicious software application 302 itself. For example, some embodiments indicate a modification of a condition following API call 308, such as condition 324. Modification of condition 324 is accomplished through various means in various embodiments. Some embodiments modify instructions contained in the potential malicious software application 302 that implement condition 324 to perform an alternative operation. Other embodiments intercept the operation of condition 324 and modify the register or memory values ​​required to change the condition outcome. As part of the modification process, the API call is allowed to return control 326 to the executor via execution modifier 320.

[0033] Figure 4 Another embodiment of a sandbox architecture 400 implemented by one or more of the disclosed embodiments is shown. The sandbox architecture 400 includes a potential malicious software application 402. The potential malicious software application 402 executes instruction streams 404, each of which is provided to a debugger 406. In some embodiments, the potential malicious software application 402 runs under the "control" of the debugger 406, which is able to "step through" each instruction executed by the potential malicious software application 402. Some other embodiments do not use... Figure 4 The debugger illustrated is used to intercept instruction stream 404. Instead, for example, some embodiments use a hardware emulator, in accordance with the above description... Figure 3 In a similar manner, the instruction stream is intercepted, resulting in a 404 error.

[0034] Instruction streams are provided by debugger 406 to instruction stream processor 408. Instruction stream processor 408 provides instruction stream 404 to machine learning model 410. In at least some embodiments, machine learning model 410 implements reinforcement learning algorithms. Model 410 includes data defining the relationship between previously suggested modifications and the successful or failed execution of a program applying the suggested modifications, as well as algorithms for determining modifications based on inputs provided to model 410, and data defining the relationships. Through sequential application by machine learning model 410, model 410 develops data defining the relationships between suggested modifications and any results obtained therefrom, and is able to use this data to provide suggested modifications 412, thereby maximizing the successful execution of potential malware applications 402.

[0035] As described above, some embodiments define successful execution as causing the potential malicious application 402 to establish a network connection with a remote device and / or generate calls to at least one additional process or thread (different from the malicious process / thread itself). Various embodiments may also use one or more file I / O activities, the creation and / or writing of system registry entries, and / or the number of API calls made by the potential malicious application to determine whether the malicious application has executed successfully.

[0036] The proposed modification 412 indicates various modifications in various embodiments. For example, in some cases, the proposed modification 412 indicates a modification of a register or memory value at a specific location within the execution of instruction stream 404. In some cases, the proposed modification 412 indicates a modification of one or more instructions contained in instruction stream 404. For example, one or more instructions are modified to change the operation of branch conditions, branches, or other conditional logic within a potential malicious software application 402.

[0037] The suggested modification 412 is provided to the executor 414, which executes the modification indicated by 416.

[0038] Figure 5 An example machine learning system 500 is shown according to some examples of this disclosure. The machine learning system 500 utilizes a prediction module 520.

[0039] In prediction module 520, current information 590 is input to feature determination module 550b. Current information 590 identifies characteristics of the potential malicious application analyzed in the disclosed embodiments (e.g., potential malicious application 302 or potential malicious application 402). Feature determination module 550b determines a set of features 570 from the current information 590. In some embodiments, this set of features includes the instruction flow of the potential malicious application, API calls of the potential malicious application, input parameters of the API calls, the call stack of the potential malicious application at the API calls, attributes of the potential malicious application, or other characteristics of the potential malicious application. The set of features 570 is provided to machine learning model 410 to generate suggested modifications 595. An indication of whether the potential malicious application operated successfully is returned to model 410 as model input 596.

[0040] Figure 6 This is a flowchart used to modify the execution of the application. In some embodiments, the following discussion concerns... Figure 6 One or more functions of method 600 are performed by a hardware processing circuit device. In some embodiments, instructions (e.g., 824) stored in a memory (e.g., memories 804 and / or 806 discussed below) configure a hardware processor (e.g., processor 802 discussed later) to perform the following... Figure 6 One or more functions discussed in Method 600.

[0041] After starting operation 602, method 600 moves to operation 604, where the executing application's instruction sequence is intercepted. (As mentioned above...) Figure 4 As discussed, in some embodiments, the instruction stream (e.g., instruction stream 404) is intercepted using debugger 406. Alternative embodiments utilize other techniques to intercept instructions, such as a hardware emulator. In some embodiments, the executing application is a "native" application, such as a first potential malware application 204, which uses the native instruction set of the hardware on which it performs its operations. In some embodiments, the executing application is an interpreted application or a script-based application, such as a second potential malware application 210, also mentioned above. Figure 2 In the discussion, the instruction sequence is not the hardware's "native" instruction, but rather intermediate instructions, such as p-code, or even source code instructions in some embodiments.

[0042] In some embodiments, the instruction sequence contains function call instructions, or in other words, API function calls.

[0043] In operation 606, a sequence of instructions is provided to a machine learning model. As described above, some embodiments use a machine learning model configured to implement a reinforcement learning method. The machine learning model is trained to maximize the likelihood of successful execution of the application in execution. In different embodiments, successful execution has different definitions. Some embodiments define successful execution as the application in execution establishing a network connection with a remote computer, creating a new process or thread, or creating or writing to a file, at least one of the following:

[0044] In operation 608, a suggested modification from the machine learning model is received. The suggested modification is in response to the instruction sequence provided in operation 606. As described above, several different modifications are suggested to the machine learning model under various conditions. In some cases, modifications are not suggested. In some cases, the machine learning model instructs modification of register values ​​or data values ​​of the executing application at a specific execution point. In some cases, the modification instructs modification of the return value or input parameters of an API function. In some embodiments, the suggested modification changes the operation of conditional statements within the executing application itself, for example, by modifying the instructions within the executing application itself, or by modifying register values ​​and / or data values ​​to change the conditional operation of the executing application.

[0045] In operation 614, the indicated modifications are executed. Some embodiments of method 600 are executed iteratively, with a single application being executed. As described above, the disclosed embodiments provide successful operations for adding potential malware to a sandbox environment. By successfully executing the potential malware, additional information about the malware can be obtained. In some embodiments, this information is used to configure runtime malware detection software, identify vulnerabilities in existing software, or further improve the sandbox testing environment. After operation 614, method 600 moves to end operation 620.

[0046] Some embodiments of method 600 maintain a log or record of modifications made to a potentially malicious application. Therefore, in at least some embodiments, a series of modifications are provided in this log. After a particular potentially malicious application is executed in a sandbox environment, some embodiments generate one or more reports that provide or display at least a portion of the data from the log. Therefore, this report identifies the modifications required for the successful execution of the potentially malicious application.

[0047] In some embodiments, this sequence of modifications is used to modify the sandbox environment to make it undetectable to other potential malware applications. For example, if modifying the API call value to a specific value frequently and successfully facilitates the successful execution of a potential malware application, the sandbox configuration is modified so that, in some cases, the API call value returns a specific value without any intervention from the disclosed embodiments. For example, if the API call is modified to return the number of processors contained in the sandbox computer, where a value of eight (8) is provided for the successful execution of a potential malware application, the operating system configuration file is modified to make the API return a value of eight (8).

[0048] Some embodiments provide algorithms that analyze modification logs and automatically (e.g., without human intervention) generate modifications to the sandbox environment to avoid the need for such modifications. For example, in some embodiments, the algorithm identifies modified return values ​​of one or more APIs and automatically changes the sandbox configuration to align with the modified return values. This reduces the number of modifications required during the dynamic execution of additional potential malware applications.

[0049] Figure 7 This is a flowchart of a method for modifying the execution of an application. In some embodiments, the following discussion concerns... Figure 7 One or more functions of method 700 are performed by a hardware processing circuit device. In some embodiments, instructions (e.g., 824 discussed below) stored in memory (e.g., memory 804 and / or 806 discussed below) configure a hardware processor (e.g., processor 802 discussed later) to perform the following... Figure 7 The methods discussed in method 700 include one or more of the functions described above. In some implementations, method 700 includes the functions discussed above. Figure 6 In method 600, for example, some embodiments of method 600 at least partially overlap with embodiments of method 700.

[0050] After initiating operation 702, method 700 moves to operation 704, which intercepts function calls of the executing application. In some embodiments, the executing application runs within a sandbox environment. A sandbox environment is a virtualized computing environment, at least in some embodiments. In at least some embodiments, the application is suspected of being a malicious application. Therefore, the application is at least likely to contain malicious (e.g., malicious) characteristics, but is not necessarily malicious. In some cases, potentially malicious applications are completely benign and harmless applications. However, some other potentially malicious applications are indeed malicious and are configured to disrupt computing resources or perform other malicious activities, such as unauthorized copying or transfer of data, ransomware, or other functions understood to be included in the term "malicious".

[0051] As mentioned above Figure 3 The embodiments discussed intercept API calls made by an executing application. In some embodiments, the executing application is a "native" application, such as a first potential malware application 204, which executes a native instruction set on which hardware operates. In some embodiments, the executing application is an interpreted application or a script-based application, such as a second potential malware application 210, also mentioned above. Figure 2 In the discussion. In some embodiments (e.g., API library 309), the called function is contained in the API library. Some embodiments of operation 704 capture or intercept a series of function calls. In these embodiments, for each captured function call, each function discussed below with respect to operations 706 and 708 is repeated.

[0052] In operation 706, the input parameter values ​​for the function call are determined. In some embodiments, the function call does not include input parameters. Some embodiments also obtain the call stack of the executing application. For example, some embodiments send signal 3 to the Java process, which causes a stack trace to be generated to the standard output (stdout) device. Some embodiments use a utility called jstack in the command line to obtain the call stack of the executing application. While these solutions apply to Java-based solutions, other solutions exist for other technologies, such as Microsoft Windows. For example, Microsoft provides debugging tools for Windows. Some embodiments of operation 706 also capture a portion of the executing application. For example, a predetermined number of bytes or a predetermined number of instructions are captured before and / or after API calls within the executing application.

[0053] In operation 708, the information captured or collected in operation 706 is provided to the machine learning model. As described above, in some embodiments, the machine learning model is configured to use a reinforcement learning algorithm. In some embodiments, the machine learning algorithm is trained to maximize the probability of successful execution of an application in execution, which in some embodiments is defined by one or more applications in execution opening a network connection to a remote device, creating a new process or thread, writing data, or otherwise creating a file.

[0054] In operation 712, the proposed modification is obtained from the machine learning model. The proposed modification is in response to or otherwise based on information provided to the machine learning model in operation 708, such as the provided call stack and input parameter values ​​of the API calls. In some embodiments, the proposed modification is based on the indicated sequence of function calls, as described above, previously provided to the machine learning model via operation 708.

[0055] As described above, several different modifications are proposed by the machine learning model under various conditions. In some cases, modifications are not recommended. In others, the machine learning model instructs modification of register or data values ​​of the executing application at specific execution points. In still others, the modification instructs modification of the return value or input parameters of an API function. In some embodiments, the recommended modification is to change the operation of conditional statements within the executing application itself, for example, by modifying the instructions themselves or by changing the conditional operation of the executing application through modifications to registers and / or data values. For example, in some embodiments, conditional branches are modified to take a first path instead of a second path.

[0056] In operation 714, the indicated modifications are executed. Some embodiments of method 700 are executed iteratively while executing a single application. As described above, the disclosed embodiments provide a successful operation for adding potential malware in a sandbox environment. By successfully executing the potential malware, further information about the malware can be obtained.

[0057] In some embodiments, this information is used to configure runtime malware detection software, identify vulnerabilities in existing software, or further improve the sandbox testing environment. For example, if a malware application is successfully executed and demonstrates the impact (negative impact) of one or more malware types via the disclosed embodiments, in some embodiments, the signature of the malicious application (e.g., a hash or checksum) is added to a security database. The security database is then compared with the signature of the executing application to detect instances of malware applications. In some embodiments, the security database is downloaded to a client device, such as a client device managed by an organization. A virus scanning filter executes on the client device and detects the executing application. The virus scanning filter then calculates the signature of the detected executing application and compares it with the signature in the security database. The virus scanning filter then detects matches between dynamically determined signatures of the executing application and compares them with one or more signatures contained in the security database. If a match is detected, the virus scanning filter suspends, aborts, or otherwise mitigates the execution of the application.

[0058] Therefore, the disclosed embodiments create several technical effects and represent several technical solutions. A first-order technical effect is that, instead of detecting the sandbox environment and executing those functions, the malware application will detect the sandbox environment and thus suppress the execution of one or more of its functions. By analyzing the execution of features (e.g., establishing network connections, deleting files, generating additional processes, etc.), the application's behavior can be analyzed and characterized more accurately. This characterization can then be used to identify other instances of malware applications in non-sandbox environments (such as operational environments).

[0059] The additional technical solutions and / or effects provided by the disclosed embodiments are increased accuracy and / or increased completeness of signature information for known malicious software applications. By facilitating more comprehensive analysis of malicious software applications in a sandbox environment, the disclosed embodiments provide enhanced insight into the execution of malicious software compared to other approaches. This increased analytical capability will result in the inclusion of additional and more accurate signature information in the security database, thereby comprehensively reducing instances of successful malicious software attacks.

[0060] After operation 714, method 700 moves to end operation 718.

[0061] Figure 8 A block diagram of an example machine 800 is shown, in which any one or more of the techniques (e.g., methods) discussed herein can be performed. In alternative embodiments, machine 800 can operate as a standalone device or connected (e.g., networked) to other machines. In a networked deployment, machine 800 can operate as a server machine, a client machine, or both in a server-client network environment. In the example, machine 800 can act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 800 is a personal computer (PC), tablet computer, set-top box (STB), personal digital assistant (PDA), mobile phone, smartphone, World Wide Web adapter, network router, switch or bridge, server computer, database, conference room device, or any machine capable of executing instructions (sequence or other) specifying the actions to be taken by that machine. In various embodiments, machine 800 can perform the actions described above. Figures 1 to 7 One or more processes. Furthermore, while only a single machine is illustrated, the term "machine" should also include any collection of machines, individually or in combination, that execute a set (or more) of instructions to perform any one or more methods discussed herein, such as cloud computing, Software as a Service (SaaS), and other computer cluster configurations.

[0062] As described herein, examples may contain or operate on logic or multiple components, modules, or mechanisms (collectively referred to as “modules”). A module is a tangible entity (e.g., hardware) capable of performing a specific operation and configured or arranged in a certain way. In the examples, circuits are arranged as modules in a specified manner (e.g., internal or relative to external entities, such as other circuits). In the examples, all or part of one or more computer systems (e.g., standalone client or server computer systems) or one or more hardware processors are configured as a module by firmware or software (e.g., instructions, application portions, or applications) to perform a specific operation. In the examples, the software may reside on a non-transitory computer-readable storage medium or other machine-readable medium. In the examples, when the software is executed by the underlying hardware of the module, it causes the hardware to perform the specific operation.

[0063] Therefore, the term "module" is understood to include tangible entities, meaning entities that, through physical construction, specific configuration (e.g., hardwiring), or temporary configuration (e.g., temporary programming), operate or perform any part or all of the operations described herein in a particular manner. Considering the example of temporarily configured modules, each module does not need to be instantiated at any given time. For example, where modules include a general-purpose hardware processor configured using software, the general-purpose hardware processor is configured as its respective different modules at different times. The software can accordingly configure the hardware processor, for example, constituting a specific module in one instance of time and different modules in instances of time.

[0064] Machine (e.g., computer system) 800 may include a hardware processor 802 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 804, and static memory 806, some or all of which may communicate with each other via interconnect 808 (e.g., a bus). Machine 800 may further include a display unit 810, an alphanumeric input device 812 (e.g., a keyboard), and a user interface (UI) navigation device 814 (e.g., a mouse). In the example, display unit 810, input device 812, and UI navigation device 814 are touchscreen displays. Machine 800 may additionally include a storage device (e.g., a drive unit) 816, a signal generation device 818 (e.g., a speaker), a network interface device 820, and one or more sensors 821, such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 800 may include an output controller 828, for example, a serial (e.g., Universal Serial Bus (USB), parallel, or other wired or wireless (e.g., Infrared (IR), Near Field Communication (NFC), etc.) connection, to communicate with or control one or more peripheral devices (e.g., printers, card readers, etc.).

[0065] Storage device 816 may include machine-readable medium 822 on which one or more sets of data structures or instructions 824 (e.g., software) are stored, which embody or be utilized by any one or more of the technologies or functions described herein. Instructions 824 may also reside wholly or at least partially in main memory 804, static memory 806, or during execution of machine 800 in hardware processor 802. For example, hardware processor 802, main memory 804, static memory 806, or storage device 816, or any combination thereof, may constitute a machine-readable medium.

[0066] Although machine-readable medium 822 is illustrated as a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions 824.

[0067] The term "machine-readable medium" can include any medium capable of storing, encoding, or carrying instructions for execution by machine 800 and enabling machine 800 to perform any one or more of the techniques disclosed herein, or any medium capable of storing, encoding, or carrying data structures for or associated with such instructions. Examples of non-limiting machine-readable media may include solid-state memory, optical, and magnetic media. Specific examples of machine-readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; disks, such as internal hard disks and removable hard disks; magneto-optical disks; random access memory (RAM); solid-state drives (SSDs); and CD-ROMs and DVD-ROMs. In some examples, a machine-readable medium may include non-transitory machine-readable media. In some examples, a machine-readable medium may include machine-readable media that is not a transient propagation signal.

[0068] Instruction 824 can be further transmitted or received on communication network 826 via network interface device 820 using a transmission medium. Machine 800 can use one or more transmission protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Example communication networks can include Local Area Networks (LANs), Wide Area Networks (WANs), Packet Data Networks (e.g., the Internet), Mobile Phone Networks (e.g., Cellular Networks), Common Telephone Service (POTS) Networks, and Wireless Data Networks (e.g., IEEE 802.11 series standards, referred to as...). IEEE 802.16 series standards (referred to as...) This includes standards such as the IEEE 80215.4 series, the Long Term Evolution (LTE) series, the Universal Mobile Telecommunications System (UMTS) series, and peer-to-peer (P2P) networks. In the example, network interface device 820 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to communication network 826. In the example, network interface device 820 may include multiple antennas to perform wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. In some examples, network interface device 820 may use multi-user MIMO technology for wireless communication.

[0069] As described herein, examples may contain or operate on logic or multiple components, modules, or mechanisms. A module is a tangible entity (e.g., hardware) capable of performing a specific operation and configured or arranged in a certain way. In the examples, circuits are arranged as modules in a specific manner (e.g., internally or relative to external entities, such as other circuits). In the examples, all or part of one or more computer systems (e.g., standalone client or server computer systems) or one or more hardware processors are configured as a module by firmware or software (e.g., instructions, application portions, or applications) to perform a specified operation. In the examples, the software may reside on a machine-readable medium. In the examples, when the software is executed by the underlying hardware of the module, it causes the hardware to perform the specific operation.

[0070] Example 1 is a method that includes: intercepting a sequence of instructions from an executing application; providing the sequence of instructions to a machine learning model; and receiving from the machine learning model instructions on modifications to the executing application based on the provided sequence of instructions.

[0071] In Example 2, the subject of Example 1 may optionally include a sequence of instructions that are machine instructions or bytecode instructions.

[0072] In Example 3, the subject of any one or more of Examples 1-2 may optionally include an instruction sequence containing function call instructions, the method further comprising: determining the call stack of the application in execution at the function call instruction; determining the parameters of the function call; and providing the call stack and parameters to a machine learning model, wherein the indication of modification is further based on the provided call stack and parameters.

[0073] In Example 4, the subject of Example 3 may optionally include determining the second parameter of the second function call and providing the second parameter of the second function call to the machine learning model, wherein the indication of modification is further based on the provided second parameter.

[0074] In Example 5, any one or more of the topics from Examples 3-4 may optionally include modifications to the output of a function call.

[0075] In Example 6, the subject of Example 5 may optionally include selecting a return value from a list of return values ​​for a function call and setting the output to the selected return value.

[0076] In Example 7, any one or more of the topics in Examples 3-6 may optionally contain control flow instructions that identify the application in execution, wherein the modification is a modification of the control flow instructions.

[0077] In Example 8, any one or more of the topics in Examples 3-7 may optionally include portions of the executable code of the application in execution within a predefined proximity of function calls, and provide these portions to the machine learning model, wherein the instructions for modification are based on the provided portions.

[0078] In Example 9, any one or more of the topics in Examples 3-8 may optionally include capturing the function call sequence of an application in execution, and providing data that indicates the function call sequence to a machine learning model, wherein the modified indication is based on the indicated function call sequence.

[0079] In Example 10, any one or more of the topics in Examples 1-9 may optionally be included, wherein a machine learning model is configured to apply a reinforcement learning algorithm, and the machine learning model is trained to generate modifications that cause successful execution of the application in execution.

[0080] In Example 11, the subject of Example 10 may optionally include the following: when an executing application creates a new process, creates a new file, creates a new registry key, establishes a network connection, or the executing application invokes more than a predefined threshold of API calls, successful execution is detected.

[0081] In Example 12, any one or more of the topics from Examples 1-11 may optionally be included in a data store that stores records indicating modifications to the application in execution.

[0082] In Example 13, the topic of Example 12 may optionally include identification based on multiple records in the data store, which is a modification shared by multiple records; and modification of the sandbox environment configuration so that the configuration is consistent with the modification of the identification.

[0083] Example 14 is a system comprising: hardware processing circuitry; one or more hardware memories storing instructions that, when executed, configure the hardware processing circuitry to perform operations, including: intercepting a sequence of instructions from an executing application; providing the sequence of instructions to a machine learning model; receiving from the machine learning model instructions for modifications to the executing application based on the provided sequence of instructions; and executing the indicated modifications.

[0084] In Example 15, the subject of Example 14 may optionally include a sequence of instructions that are machine instructions or bytecode instructions.

[0085] In Example 16, any one or more of the topics in Examples 14-15 may optionally include an instruction sequence containing a function call instruction, the operation further comprising: determining the call stack of the application in execution at the function call instruction; determining the parameters of the function call; and providing the call stack and parameters to a machine learning model, wherein the indication of modification is further based on the provided call stack and parameters.

[0086] In Example 17, the subject of Example 16 may optionally include determining a second parameter for a second function call and providing the second parameter of the second function call to the machine learning model, wherein the indication of modification is further based on the provided second parameter.

[0087] In Example 18, any one or more of the topics from Examples 16-17 may optionally include modifications to the output of a function call.

[0088] In Example 19, the subject of Example 18 optionally includes selecting a return value from a list of return values ​​for a function call and setting the output to the selected return value.

[0089] In Example 20, any one or more of the topics in Examples 16-19 may optionally contain control flow instructions that identify the application in execution, wherein the modification is a modification of the control flow instructions.

[0090] In Example 21, any one or more of the topics in Examples 16-20 may optionally include portions of the executable code of the application in execution within a predefined proximity of function calls, and provide these portions to a machine learning model, wherein the instructions for modification are based on the provided portions.

[0091] In Example 22, any one or more of the topics in Examples 16-21 may optionally include capturing the function call sequence of an application in execution, and providing data to a machine learning model that indicates the function call sequence, wherein the modified indication is based on the indicated function call sequence.

[0092] In Example 23, any one or more of the topics in Examples 14-22 may be optionally included, wherein the machine learning model is configured to apply a reinforcement learning algorithm, and the machine learning model is trained to generate modifications that cause successful execution of the application in execution.

[0093] In Example 24, the subject of Example 23 may optionally include the following: successful execution is detected when an executing application creates a new process, creates a new file, creates a new registry key, establishes a network connection, or invokes more than a predefined threshold of API calls.

[0094] In Example 25, any one or more of the topics from Examples 14-24 may optionally be included in a data store that stores records indicating modifications to the application in execution.

[0095] In Example 26, the topic of Example 25 may optionally include modifications shared by multiple records based on multiple record identifiers in the data store; and modifications to the sandbox environment configuration so that the configuration is consistent with the modifications to the identifiers.

[0096] Example 27 is a non-transitory computer-readable storage medium containing instructions that, when executed, configure hardware processing circuitry to perform operations, including: intercepting a sequence of instructions of an executing application; providing the sequence of instructions to a machine learning model; receiving from the machine learning model instructions for modifications to the executing application based on the provided sequence of instructions; and performing the indicated modifications.

[0097] In Example 28, the subject of Example 27 may optionally include a sequence of instructions that are machine instructions or bytecode instructions.

[0098] In Example 29, any one or more of the topics in Examples 27-28 may optionally include an instruction sequence containing a function call instruction, the operation further comprising: determining the call stack of the application in execution at the function call instruction; determining the parameters of the function call; and providing the call stack and parameters to a machine learning model, wherein the indication of modification is further based on the provided call stack and parameters.

[0099] In Example 30, the subject of Example 29 may optionally include determining a second parameter of a second function call and providing the second parameter of the second function call to a machine learning model, wherein the indication of modification is further based on the provided second parameter.

[0100] In Example 31, any one or more of the topics in Examples 29-30 may optionally include modifications to the output of a function call.

[0101] In Example 32, the subject of Example 31 may optionally include selecting a return value from a list of return values ​​for a function call and setting the output to the selected return value.

[0102] In Example 33, any one or more of the subjects in Examples 31-32 may optionally contain control flow instructions that identify the application in execution, wherein the modification is a modification of the control flow instructions.

[0103] In Example 34, any one or more of the topics in Examples 31-33 may optionally include portions of the executable code of the application in execution within a predefined proximity of function calls, and provide these portions to a machine learning model, wherein the instructions for modification are based on the provided portions.

[0104] In Example 35, any one or more of the topics in Examples 31-34 may optionally include capturing the function call sequence of an application in execution, and providing data indicating the function call sequence to a machine learning model, wherein the modified indication is based on the indicated function call sequence.

[0105] In Example 36, any one or more of the topics in Examples 27-35 may optionally be included, wherein the machine learning model is configured to apply a reinforcement learning algorithm, and the machine learning model is trained to generate modifications that cause the application to be successfully executed.

[0106] In Example 37, the subject of Example 36 may optionally include the following: successful execution is detected when an executing application creates a new process, creates a new file, creates a new registry key, establishes a network connection, or invokes more than a predefined threshold of API calls.

[0107] In Example 38, any one or more of the topics from Examples 27-37 may optionally be included in a data store that stores records indicating modifications to the application in execution.

[0108] In Example 39, the topic of Example 38 may optionally include modifications shared by multiple records based on multiple record identifiers in the data store; and modifications to the sandbox environment configuration so that the configuration is consistent with the modifications to the identifiers.

[0109] Therefore, the term "module" is understood to include tangible entities, meaning entities whose physical construction, specific configuration (e.g., hardwiring), or temporary configuration (e.g., temporary programming) operates or performs any part or all of the operations described herein in a particular manner. Considering the example of temporarily configured modules, each module does not need to be instantiated at any given time. For example, if a module contains a general-purpose hardware processor configured using software, the general-purpose hardware processor is configured as different modules at different times. The software can accordingly configure the hardware processor, for example, constituting a specific module at one point in time and different modules at different points in time.

[0110] Various embodiments are implemented wholly or partially in software and / or firmware. This software and / or firmware may take the form of instructions contained in or on a non-transitory computer-readable storage medium. These instructions may then be read and executed by one or more processors to perform the operations described herein. These instructions may exist in any suitable form, such as, but not limited to, source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Such computer-readable medium may comprise any tangible non-transitory medium for storing information in one or more computer-readable forms, such as, but not limited to, read-only memory (ROM); random access memory (RAM); disk storage media; optical storage media; flash memory, etc.

Claims

1. A system comprising: hardware processing circuitry; and one or more hardware memories storing instructions that, when executed, configure the hardware processing circuitry to perform operations comprising: intercepting a sequence of instructions to be executed by an in-execution application; providing the sequence of instructions to a machine learning model; receiving, from the machine learning model, an indication of a modification to the instructions based on the provided sequence of instructions; performing the indicated modification to the instructions while the potentially malicious application is executing in a controlled analysis environment to obtain modified instructions; and receiving an indication that execution of the modified instructions results in successful execution of the in-execution application when the in-execution application performs one of: creating a new process, creating a new file, creating a new registry entry, establishing a network connection, or the in-execution application invokes more than a predefined threshold of API calls.

2. The system of claim 1, wherein the in-execution application is a potentially malicious application executing in a sandboxed environment.

3. The system of claim 1, wherein the sequence of instructions contains a function call instruction, the operations further comprising: determining a call stack of the in-execution application at the function call instruction; determining parameters of the function call; and providing the call stack and the parameters to the machine learning model, wherein the indication of the modification is further based on the provided call stack and parameters. determining second parameters of a second function call, and providing the second parameters of the second function call to the machine learning model, wherein the indication of the modification is further based on the provided second parameters.

4. The system of claim 3, further comprising:

5. The system of claim 3, wherein the modification is a modification to an output of the function call. selecting a return value from a list of return values of the function call, and setting the output to the selected return value.

6. The system of claim 5, further comprising: identifying a control flow instruction of the in-execution application, wherein the modification is a modification to the control flow instruction.

7. The system of claim 3, further comprising: capturing a portion of executable code of the in-execution application within a predefined proximity of the function call, and providing the portion to the machine learning model, wherein the indication of the modification is based on the provided portion.

8. The system of claim 3, further comprising: capturing a sequence of function calls of the in-execution application, and providing data indicative of the sequence of function calls to the machine learning model, wherein the indication of the modification is based on the indicated sequence of function calls.

9. The system of claim 3, further comprising:

10. The system of claim 1, wherein the machine learning model is configured to apply a reinforcement learning algorithm, the machine learning model trained to generate modifications that result in successful execution of the in-execution application. storing a record in a data store, the record indicating the modification to the instructions.

11. The system of claim 1, further comprising:

12. The system of claim 11, further comprising: based on a plurality of records in the data store, identifying a modification common to the plurality of records; and ​ modify a sandbox environment configuration so that the configuration is consistent with the identified modification.

13. A method of detonating a potentially malicious software application in a sandbox environment, comprising: intercepting a sequence of instructions to be executed by a potentially malicious software application while the potentially malicious software application is executing in a sandbox environment; providing the sequence of instructions to a machine learning model; receiving, from the machine learning model, an indication of a modification to the instructions based on the provided sequence of instructions; performing the indicated modification to the instructions while the potentially malicious software application is executing in the sandbox environment to obtain modified instructions; and receiving an indication that execution of the modified instructions results in successful execution of the potentially malicious software application, the indication being that the potentially malicious software application performs one of the following: creates a new process, creates a new file, creates a new registry entry, establishes a network connection, or an application in the execution invokes more than a predefined threshold of API calls.

14. The method of claim 13, wherein the sequence of instructions includes a function call instruction, the method further comprising: determining a call stack of the potentially malicious software application at the function call instruction; determining parameters of the function call; and providing the call stack and the parameters to the machine learning model, wherein the indication of the modification is further based on the provided call stack and parameters. determining a second parameter of a second function call, and providing the second parameter of the second function call to the machine learning model, wherein the indication of the modification is further based on the provided second parameter.

16. The method of claim 14, wherein the modification is a modification to an output of the function call.

15. The method of claim 14, further comprising: identifying a control flow instruction of the potentially malicious software application, wherein the modification is a modification to the control flow instruction. capturing a portion of executable code of the potentially malicious application within a predefined proximity of the function call, and providing the portion to the machine learning model, wherein the indication of the modification is based on the provided portion.

17. The method of claim 14, further comprising: capturing a sequence of function calls of the potentially malicious application, and providing data indicative of the sequence of function calls to the machine learning model, wherein the indication of the modification is based on the indicated sequence of function calls.

18. The method of claim 14, further comprising:

20. A non-transitory computer-readable storage medium comprising instructions that, when executed, configure hardware processing circuitry to perform operations comprising:

19. The method of claim 14, further comprising: intercepting a sequence of instructions to be executed by a potentially malicious software application executing in a sandbox environment; providing the sequence of instructions to a machine learning model; receiving, from the machine learning model, an indication of a modification to the instructions based on the provided sequence of instructions; performing the indicated modification to the instructions while the potentially malicious software application is executing in the sandbox environment; and ​ ​ ​ Receiving an indication of successful execution of the potentially malicious application as a result of the execution of the modified instructions, the indication being that the potentially malicious application performed one of the following: created a new process, created a new file, created a new registry entry, established a network connection, or the executing application invoked more than a predefined threshold of API calls.

Images