Function identity prediction

Abstract

This disclosure relates to predicting an identity of a function called within assembly language instructions. A processor identifies, in the assembly language instructions, a function call instruction to call the function; determines one or more input parameters of the function based on the assembly language instructions preceding the function call instruction; and creates a subset of the assembly language instructions comprising one or more of the assembly language instructions linked to the one or more input parameters. The processor creates a context input for a trained machine learning model comprising the subset of the assembly language instructions; and evaluates the trained machine learning model on the context input to generate an output corresponding to a predicted identity of the function.

Description

"Function Identity Prediction"Cross-Reference to Related Applications

[0001] The present application claims priority from Australian Provisional Patent Application No 2024903444 filed on 23 October 2024, the contents of which are incorporated herein by reference in their entirety.Technical Field

[0002] This disclosure relates to predicting an identity of a function called within assembly language instructions.Background

[0003] Source code written in high-level programming languages like C, C++, or Java can be compiled into a software file, such as a binary file, so that the computer can directly run the corresponding software. A binary file (which may be referred to simply as a binary) refers to a compiled executable file containing machine code, which is a list of instructions that the processor of a computer (e.g., the central processing unit (CPU)) can execute directly by starting at the beginning and performing each instruction at a time.

[0004] While the original source cannot be obtained from a binary, the binary may be converted into programming language instructions through a decompiler, for example. However, due to the one-way nature of the compilation process, decompilers usually cannot perfectly recreate the original source code. In another example, the binary may be converted into assembly language instructions by way of a disassembler program (which may be referred to simply as a disassembler). The disassembler essentially converts the binary code for instructions into human readable text for each instruction. Examples for assembly instructions are LDA A for loading register A, STA A for storing in register A, ADD for addition, CMP for comparison and so on. Assembly language instructions are low-level commands that are closely related to the machine code instructions executed by a computer processor. However, similar to decompilers, a disassembler program may not perfectly create assembly language instructions from the corresponding binary.

[0005] Assembly language instructions may provide an indication of software functions, such as application programming interfaces (APIs), utilised by the software, for example. More particularly, the instruction set comprises a JMP command that causes the CPU to jump to a different memory address in the list of instructions, changing the program’s control flow. This address could be the beginning of the definition of a function, a loop, a conditional branch, or another part of the code. That function may be part of an API if they are provided by the operating system or another third-party software package, allowing interaction with libraries or services. The identity of the function may be the label that is given to that address where the function starts, or the name of the function in source code before compilation.

[0006] Determining which functions are utilised in a software program can be useful for many security applications including malware analysis, capability analysis, vulnerability identification, reverse engineering, and compatibility testing. However, assembly language instructions do not always provide the true function name to accurately identify the corresponding function. In particular, malware authors may attempt to complicate the analysis of assembly language instructions by obfuscating the names of functions used in a binary to make it difficult for security analysts and anti-malware scanners to analyse and detect the malware. Obfuscation is a prominent technique used by malicious entities to evade detection and subsequent analysis, which involves obscuring the name of a function (such as an API) in source code to evade detection by malware detection software or otherwise. Obfuscation techniques may include loading functions at runtime, the deletion or modification of Import Address Table (IAT), function hashing, string encryption and name mangling.

[0007] Obfuscation may replace the function names with placeholders, such as dummy or meaningless words, which are either determined at runtime or through reverse engineering the obfuscation technique to restore the function names. Fig. 1 shows a real-world example of such obfuscation, illustrating both the standard and obfuscated disassembly for the CreateFileW API function. As can be seen, in the obfuscated case, the API name is a placeholder with a dummy name dword 4031 A5. Due to the obfuscation, the software may not be detected by malware detection scanners, which may cause undesired consequences for a user or computer, if the software were to be executed. Hence, there is a need for accurately predicting function names called within assembly language instructions to address obfuscation.

[0008] Some approaches may utilise the original source code of the compiled binary to provide information about functions used within the code to identify the function. However, on someoperation systems, such as the Windows Operating System (OS), the original source code is either unavailable or difficult to obtain. For example, WinAPI functions (e.g., CreateProcess, WinExec) are pre-built and shipped with Windows OS. There are more than 4,000 such WinAPI functions, and their source code is not available. Some malware does not call these functions directly; instead, it obfuscates their use, for example, by calling WinAPI functions only at runtime or by employing API-hashing techniques, because direct calls can be detected by antimalware tools such as Microsoft Defender.

[0009] Moreover, these approaches, even on simple, non-obfuscated binaries, are limited and inadequate for binaries that employ sophisticated obfuscation methods. Some operating systems, such as Windows OS, also comprise a large number of functions with the number of input parameters for these functions ranging from 0 to 14. Given the large diversity in functions and corresponding input parameters, accurately predicting an identity of a function utilised in a binary can be difficult. Some approaches are also rules-based, which rely on predetermined rules based on known-existing malicious functions. However, these approaches may be vulnerable to zero-day attacks.

[0010] Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present disclosure as it existed before the priority date of each of the appended claims.

[0011] Throughout this specification the word “comprise”, or variations such as “comprises” or “comprising”, will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps.Summary

[0012] Disclosed herein are a method and system for predicting an identity of a function called within assembly language instructions. In particular, the disclosed method and system utilise a machine learning approach to predict an identity of a function, where contextual information is extracted from the assembly language instructions and used as input to a machine learning model to accurately predict the identity of a function called within assembly language instructions.

[0013] According to a broad aspect of the present disclosure, there is provided a method for predicting an identity of a function called within programming language instructions derived from a compiled software file, the method comprising:identifying, in the programming language instructions, a function call instruction to call the function;determining one or more input parameters of the function based on the programming language instructions preceding the function call instruction;creating a subset of the programming language instructions comprising one or more of the programming language instructions linked to the one or more input parameters;creating a context input for a trained machine learning model comprising the subset of the programming language instructions; andevaluating the trained machine learning model on the context input to generate an output corresponding to a predicted identity of the function.

[0014] According to the present disclosure, there is provided a method for predicting an identity of a function called within assembly language instructions, the method comprising: identifying, in the assembly language instructions, a function call instruction to call the function;determining one or more input parameters of the function based on the assembly language instructions preceding the function call instruction;creating a subset of the assembly language instructions comprising one or more of the assembly language instructions linked to the one or more input parameters;creating a context input for a trained machine learning model comprising the subset of the assembly language instructions; andevaluating the trained machine learning model on the context input to generate an output corresponding to a predicted identity of the function.

[0015] It is an advantage to create a context input comprising one or more of the assembly language instructions linked to the one or more input parameters as this provides contextual information regarding how the function is utilised within the assembly language instructions. As such, the trained machine learning model may accurately predict the identity of the function given the contextual information provided in the context input. This is advantageous when malicious function names are obfuscated.

[0016] In some embodiments, the subset of the assembly language instructions comprises a first subset, each of the first subset comprising an operand corresponding to at least one of the one or more input parameters.

[0017] In some embodiments, the subset of the assembly language instructions comprises a second subset, each of the second subset being linked to one or more of the first subset.

[0018] In some embodiments, each of the second subset comprises an operand corresponding to a shared parameter, wherein at least one of the assembly language instructions preceding the function call instruction comprises the shared parameter and at least one of the one or more input parameters.

[0019] In some embodiments, the assembly language instructions define an instruction order, and the method comprises determining the subset of the assembly language instructions by considering the assembly language instructions in reverse instruction order.

[0020] In some embodiments, the context input comprises parameter name strings of the one or more input parameters.

[0021] In some embodiments, the subset of the assembly language instructions further comprises one or more external functions called within the assembly language instructions.

[0022] In some embodiments, creating the context input comprises creating a symbolic string for one or more of the subset of the assembly language instructions.

[0023] In some embodiments, creating the symbolic string comprises creating a symbolic string for one or more of the subset of the assembly language instructions corresponding to one or more of: a memory address; a pointer address; a complex expression; a hexadecimal value representing a location with the assembly language instructions; and an external function.

[0024] In some embodiments, the method further comprises receiving a complied software file and applying a disassembler to the complied software file to generate the assembly language instructions.

[0025] In some embodiments, the complied software file is a binary file.

[0026] In some embodiments, the trained machine learning model is at least partially trained by tokenising the context input and randomly masking one or more tokens of the tokenised context input.

[0027] In some embodiments, the context input comprises the function call instruction and the trained machine learning model is at least partially trained by masking a token corresponding to the function call instruction.

[0028] In some embodiments, the trained machine learning model is at least partially trained by using training data comprising one or more of: a malicious function; and an obfuscated function.

[0029] In some embodiments, the trained machine learning model is fine-tuned after at least partial training by using training data comprising at least one stripped set of assembly language instructions.

[0030] In some embodiments, the trained machine learning model is trained to generate output text and evaluating the trained machine learning model on the context input generates an output text corresponding to a predicted identity of the function.

[0031] In some embodiments, the one or more input parameters correspond to one or more parameters being pushed before the function call instruction.

[0032] In some embodiments, the function is an application programming interface (API).

[0033] According to the present disclosure, there is provided software that, when executed by a computer, causes the computer to perform the method of any one of the previously mentioned embodiments.

[0034] According to the present disclosure, there is provided a system for predicting an identity of a function called within assembly language instructions, the system comprising one or more processors configured to perform the method of any one of the previously mentioned embodiments.

[0035] Optional features provided in relation to the method, equally apply as optional features to the software and the system.Brief Description of Drawings

[0036] An example will be described with reference to the following drawings:

[0037] Fig. 1 illustrates a real-world example of the standard and obfuscated disassembly for the CreateFileW API function.

[0038] Fig. 2 illustrates a system for predicting an identity of a function called within assembly language instructions.

[0039] Fig. 3 illustrates a method for predicting an identity of a function called within assembly language instructions.

[0040] Fig. 4 shows a real-world example of the API call sendto demonstrating the API parameters push onto the stack.

[0041] Fig. 5a shows an overview of an example embodiment of the method disclosed herein.

[0042] Fig. 5b shows a continuation of the overview shown in Fig. 5a.

[0043] Fig. 6 shows construction of a context input through a backtracking process.

[0044] Fig. 7 shows an example of the FindResourceA API under normal and stripped binary cases.

[0045] Fig. 8a presents the distribution of confidence scores for the number of parameters in 4,123 Windows APIs.

[0046] Fig. 8b presents the distribution of confidence scores for the names of parameters in 4,123 Windows APIs.

[0047] Fig. 9 illustrates the distribution (logarithmic scale) of 3,241 unique APIs within the training dataset described herein.

[0048] Fig. 10 shows a histogram of API prediction accuracy.

[0049] Fig. 11 shows the top 30 most frequently used APIs in malware with respective prediction accuracies.Description of Embodiments

[0050] Disclosed herein are a method and system for predicting an identity of a function called within assembly language instructions. In particular, the disclosed method provides a function name prediction framework that learns different contextual representations of API functions, present in binaries and uses this knowledge for accurate function name prediction in obfuscated binaries. It is noted that the function name is one example of an “identity of a function called within assembly language instructions”. Other examples may relate to a label in the assembly code, a known address for that functions or other aspects that can serve to uniquely identify that function.

[0051] The disclosed method generates a context input, which may be referred to as a “codeprinf ’ in this disclosure, for a trained machine learning model, which provides contextual information based on linked assembly language instructions.

[0052] Function names may be determined by examining the arguments passed to them and the context in which they are invoked. For instance, as illustrated in Fig. 1, the parameters dwDesiredAccess, dwShareMode, dwCreationDisposition, and dwFlagsAndAttributes are the pre-defined constants indicating access control parameters, file sharing restrictions and attributes, and permission flags, respectively. They can take only a small set of potential values. For instance, the dwCreationDisposition and dwShar eMode parameters only accept five and four pre-defined constant values, respectively, ranging from 1 to 5 and 1 to 4. As such, these parameters on the stack may conclude that the CreateFileW API is called, even if the function name is not explicitly visible in the assembly language instructions.

[0053] As such, the disclosed method relies on program analysis to extract the context of a function called within the assembly language instructions by identifying its input parameters and the corresponding assembly instructions that are linked to the input parameter, to create a context input for a trained machine learning model. The context input may consist of the function name extracted from the assembly language instructions (which may be obfuscated), names, and corresponding values of its input parameters, and a set of linked assembly instructions for each input parameter value. As will be shown later in the disclosure, the context input with the linkedassembly language instructions provides greater contextual information, leading to improved accuracy and addressing issues in predicting an identity of a function.

[0054] In some embodiments, the disclosed method may utilise a masked machine learning model to learn codeprints for functions in unsupervised manner. The machine learning task may involve predicting values of randomly masked tokens within tokenized API codeprints, compelling the model to infer missing tokens based on contextual cues from adjacent tokens. Specifically, the names of function may be inferred for a given codeprint (e.g., context input) by masking-out the actual name of the function in the codeprint. As will be shown, this context-aware understanding of the problem domain improves not only the model’s accuracy on regular binaries but also on stripped and obfuscated binaries with minimal fine tuning. Another benefit of this masked machine learning approach is that the masked token prediction task is selfsupervised, reducing the necessity of any labelled data. Thus, the disclosed method can be further improved for other downstream tasks, such as API prediction for stripped binaries, API parameters’ values prediction, or API call sequence prediction.

[0055] The disclosed method may be used for executable binaries where no corresponding source code is available. For example, the disclosed method may work on Windows binaries regardless of whether they are natively compiled (C / C++, Rust, Go, Delphi, assembly, etc.) or runtime-backed (.NET, JVM, Python bundles, etc.). The disclosed method may also detect OS interactions that are using WinAPIs from disassembly, regardless calls to those WinAPI functions are visible in the Import Address Table (e.g., CreateFileW, VirtualAlloc) or are resolved dynamically at runtime (obfuscation). The disclosed method may not only predict obfuscated WinAPI function names but also the sequence of WinAPI calls, e.g., given a pattern of API function calls, it can predict the next function likely to be invoked if fine-tuned appropriately. Similarly, it can predict API function arguments without fine-tuning.

[0056] It is noted that the disclosed method is particularly applicable for predicting the identity of an API called within assembly language instructions. In particular, “prediction” and variations thereof (e.g., predicting, predicts, predicts) in the context of the present disclosure generally refers to determining the identity of a function within called within assembly language instructions. “Prediction” and variations thereof may have a similar meaning to estimation, inference, determine, or the like. Those skilled in the art will appreciate that “prediction” and variations thereof may refer to the inference performed by the trained machine learning model,according to the present disclosure. “Prediction” and variations thereof in the context of the present disclosure should not be interpreted as corresponding to a future event or state of affairs.

[0057] It is also noted that the disclosed method is not limited to predicting the identity of an API called within assembly language instructions. The disclosed method is applicable to predicting any function called within assembly language instructions. However, for explanatory purposes, the disclosed method will be explained for the purpose for predicting the identity of an API called within assembly language instructions.

[0058] Further, while this disclosure generally discusses predicting an identity of a function called within assembly language instructions, the method disclosed herein may be equally applicable to predicting an identity of a function called within other programming language instructions. These programming language instructions may include instructions formatted in programming languages such as, but not limited to, C, C++, C#, FORTRAN, Python, JavaScript etc. In some examples, the programming language instructions may be derived from a compiled software file, such as a binary file. For example, a decompiler program may be use to determine the programming language instructions from the compiled software file. This is advantageous when the original source code for a compiled software file is not available.

[0059] However, due to the one-way nature of the compilation process, decompilers usually cannot perfectly recreate the original source code. As such, the disclosed method may be used to predict the identity of a function in programming language instructions, rather than relying on imperfectly recreated source code from the decompiler for function identification. Some functions in the original source code may also be obfuscated, which may make it more difficult to identify a function from programming language instructions derived from a compiled software file (via a decompiler, for example). As such, the disclosed method may identify the function regardless of whether it was obfuscated in the original source code.System

[0060] Fig. 2 illustrates an example embodiment of a system (denoted as system 200) for predicting an identity of a function called within assembly language instructions. Fig. 2 is one example of a configuration of system 200. However, system 200 is not strictly limited to this configuration and this may be one possible embodiment of system 200. It is noted that system200 of Fig. 2 is only meant to illustrate an example system which is capable of performing the disclosed methods.

[0061] System 200 comprises device 210, which may be smartphone, computer, tablet, a server device, or any other similar device. Device 210 may also be a field-programmable gate array (FPGA), an application specific integrated circuits (ASIC), or one or more single board computers, such as a Raspberry Pi or an Arduino. Device 210 comprises processor 211, which may be configured to perform the methods described in this disclosure. Device 210 comprises memory 212, which comprises non-volatile memory 213 and / or volatile memory 214. Processor 211 may communicate with memory 212 by communicating with non-volatile memory 213 and / or volatile memory 214. Non-volatile memory 213 is a non-transitory computer readable medium and may be an optical disk drive, hard disk drive, solid-state drive, flash memory, storage server, cloud storage or another equivalent type of memory. Volatile memory 214 may be cache, RAM, or another equivalent type of memory.

[0062] Memory 212 may store data to be retrieved for later use. For example, memory 212 may store software applications (such as binary files, for example), assembly language instructions, strings, prompts for a machine learning model and the output of the machine learning model. In essence, memory 212 may store any data used by processor 211 when performing the methods described herein. The data thereof may be stored in memory 212 in the form of a Portable Executable (PE) format file, Executable and Linkable Format (ELF) file format, JSON format file, XML format file, TXT format file or another equivalent data format file.

[0063] The methods described herein comprise applying one or more trained machine learning models. These machine learning models may be stored on memory 212 by storing the weights that form the respective models, for example. Memory 212 may also store any output calculated by processor 211 applying the one or more machine learning models, or any other variable or data necessary to perform such methods described herein.

[0064] Software, that is, an executable program stored on non-volatile memory 213 causes processor 211 to perform the method disclosed herein. While the singular of “processor” is used herein, it is meant to also encompass multiple processors that are individually or together configured (e.g., programmed) to perform the methods disclosed herein. As such, processor 211 may refer to multiple central processing units (CPUs) and / or graphical processing units (GPUs) that are configured to collectively perform the methods disclosed herein.

[0065] Once executed, the software may cause processor 211 to identify, in the assembly language instructions, a function call instruction to call the function, determine one or more input parameters of the function, create a subset of the assembly language instructions, create a context input for a trained machine learning model, and evaluate the trained machine learning model on the context input.

[0066] Software may provide a user interface (such as a graphical user interface) presented to the user on device 210. The user interface may be configured to accept input (via buttons or text fields etc) from the user, via a touch screen or a device attached to device 210 such as a keyboard or computer mouse. These devices may also include a touchpad, an externally connected touchscreen, a joystick, a button, and a dial. In an example, the user interface may display multiple sets of assembly language instructions, and a user may choose one of the sets by interacting with the user interface. The user interaction may then cause processor 211 to perform a method for predicting an identity of a function called within assembly language instructions on the chosen set of assembly language instructions.

[0067] System 200 further comprises server 220, which may be an external database, for example. Processor 211 may establish a communication with server 220 using I / O port 215 through a wireless connection, such as by using a Wi-Fi network according to IEEE 802.11. The Wi-Fi network may be a decentralised ad-hoc network, such that no dedicated management infrastructure, such as a router, is required or a centralised network with a router or access point managing the network. Processor 211 may also establish a communication with server 220 using I / O port 215 through a wired connection, such as Ethernet.

[0068] Server 220 may run an application in which a user can input a software application and causes server 220 to perform the method disclosed herein to predict an identity of a function called within assembly language instructions. Server 220 may run the backend of this application, where the frontend corresponds to a graphical user interface which is displayed on device 210, for example. Server 220 may also run any one of the machine learning models described herein. As such, a user may upload assembly language instructions to device 210, processor 211 may then create a context input for a trained machine learning model. Processor 211 may then communicate the context input to server 220, where server 220 evaluates the trained machine learning model on the context input to generate an output. Server 220 may then communicate the output to processor 211. Other variations of this approach are also possible.Method

[0069] Fig. 3 illustrates an example embodiment of a method (denoted as method 300) for predicting an identity of a function called within assembly language instructions. Fig. 3 is to be understood as a blueprint for a software program and may be implemented step-by-step, such that each step in Fig. 3 is represented by a function in a programming language, such as, but not limited to, Python, C++, or Java. The resulting source code is then compiled and stored as computer-executable instructions on non-volatile memory 214, which causes processor 211 (or multiple processors or a distributed computing architecture) to perform method 300. Preferably, the identity of the function is an associated name of the function, which may describe the use or capability of the function. “Identity” in the present context may also be a malicious identity (e.g., malicious or non-malicious). As such, the disclosed method may classify the function as being malicious or non-malicious, for example.

[0070] Processor 211 identifies 301, in the assembly language instructions, a function call instruction to call the function. The function call instruction may be one of the assembly language instructions that calls the function being identified by the disclosed method. For example, the function call instruction may be “call dword 4031 A5” , where the function name in the assembly language instructions is "dword 4031A5” . However, this may be an obfuscated name and may not be the true name of the function. The disassembler may have provided the incorrect name when converting a binary to the assembly language instructions. In some examples, processor 211 may forward scan through the assembly language instructions of each function and searches for “call” instructions.

[0071] Processor 211 determines 302 one or more input parameters of the function based on the assembly language instructions preceding the function call instruction. For example, the one or more input parameters may correspond to one or more parameters being pushed before the function call instruction. More specifically, in the assembly language instructions, one or more instructions preceding the function call instruction may be a push instruction and may be represented by “push ecx”. In this example, “ecx” may be the register that holds an input parameter value of the function. The push instruction, or similar instructions such as mov, depending on the Instruction Set Architecture (ISA), pushes the parameter on the stack to be used by the function and hence, the corresponding instruction may be used to determine the one or more input parameters. In other examples, processor 211 may receive data indicative of the one or more input parameters. For example, such data may be stored on server 220 andtransmitted to processor 211, such that processor 211 can determine the one or more parameters without searching from push instructions in the assembly language instructions.

[0072] Processor 211 creates 303 a subset of the assembly language instructions comprising one or more of the assembly language instructions linked to the one or more input parameters. “Linked” in the present context may refer to being related in some manner, or similar, or associated, or alike. In some examples, “linked” may refer to being semantically related, in the sense that there is a semantic relationship between the subset of assembly language instructions and the one or more parameters. In some examples, the subset of the assembly language instructions may be linked to the one or more input parameters as the subset comprises at least one of the one or more input parameters in the instruction. For example, an input parameter may be “ecx” and one of the subset may be the instruction “mov ecx [ebp+var_8]”. The instruction is linked to the input parameter as it comprises the input parameter within the instruction.Preferably, the subset of the assembly language instructions precedes the function call instruction in the assembly language instructions.

[0073] Processor 211 creates 304 a context input for a trained machine learning model comprising the subset of the assembly language instructions. “Context input” used in this disclosure may refer to an input of the trained machine learning model. As such, “context input” may be refer to as “input”. The “context input” may also be referred to or indicative of a “prompt” and hence, in some examples, the context input may be referred to as a “context prompt”. It is noted that the context input recited in this disclosure is not necessarily in the form of natural language. The context input may simply provide the contextual information for the trained machine learning model to generate a response or perform a task.

[0074] In some examples, the context input may be a flattened list of the subset of the assembly language instructions. More specifically, the context input may be a text string comprising each of the subset of assembly language instructions. For example, each of the subset of the assembly language instructions may be separated by a space in the context input. The subset of the assembly language instructions may be ordered in the context input in the order they appear in the assembly language instructions or a different order. In some examples, the subset of the assembly language instructions may be ordered randomly. The context input may resemble the following text string “push eax lea eax complex mov edx 3”.

[0075] Creating 303 a subset of the assembly language instructions comprising one or more of the assembly language instructions linked to the one or more input parameters provides a higher density of contextual information for the trained machine learning model, as opposed to simply creating an input comprising all of the assembly language instructions. In some examples, there may be a large amount of assembly language instructions and hence, the trained machine learning model may not be able to provide an accurate output when applied to an input comprising all of the assembly language instructions. Therefore, the context input of the disclosed method enables the trained machine learning model to provide an accurate output indicative of the identity of the function. In other words, the context input “focusses” the attention of the trained machine learning model on salient features to accurately predict the function identity. In some embodiments, the subset of the assembly language instructions further comprises one or more external functions called within the assembly language instructions. This may further increase the amount of contextual information to further increase prediction accuracy.

[0076] Processor 211 evaluates 305 the trained machine learning model on the context input to generate an output corresponding to a predicted identity of the function. The output may be output text, a numerical vector or numerical value. For example, if the output is output text, the predicted identity of the function, such as the function name, may be explicitly recited in the output text. In some examples, the output may resemble the input. For example, the output may be output text that is similar to the context input. In the case of a numerical vector, processor 211 may apply a further trained machine learning model to the numerical vector to provide a further output indicative of the predicted identify of the function. More specifically, processor 211 may apply a classifier model (such as a neural network) to the output which may classify the function into one or more classes indicative of the identity of the function. In other embodiments, the trained machine learning model may be a classifier model (such as a neural network) and the output of 305 may be a numerical value indicative of the predicted identity of the function.

[0077] Each of the machine learning models described herein are understood to be a model, such as a mathematical model, that receives input text (such as the “context input”) or other inputs such as, but not limited to, numerical values. The machine learning models may be of an architecture, such as, but not limited to, a neural network or a transformer model, for example. In general, machine learning models are ‘trained’ to learn and recognise patterns in an input and provide an output that is a prediction based on the training it has undergone. Training may involve updating weights or parameters (as referred to as hyperparameters) of the machinelearning model, which define the machine learning model, to minimise a loss value, thereby creating a trained machine learning model (in other words, a machine learning model trained to generate an output). This may involve a gradient descent and backpropagation method.

[0078] The machine learning models may be stored on memory 212 or server 220 by storing the weights that define the model. As such, the machine learning models may be referred to as a “memory model”, given that it is defined by parameters (i.e., the weights) which can be stored on computer memory. In some embodiments, the machine learning models may be programmed on an integrated circuit, such as a field-programmable gate array (FPGA) or an NVIDIA processing unit. In such an embodiment, processor 211 may not retrieve the parameters from data memory 104. Instead, an input may be communicated from processor 211 to the integrated circuit and the integrated circuit may apply the machine learning model to the input and generate an output, which is then communicated to processor 211.

[0079] Integrated circuits, such as FPGAs, can be used where flexibility, speed, and parallel processing capabilities are desired. In such an embodiment, the integrated circuit may be part of device 210 of system 200 and may be considered as a “processor” or “processing unit”, similar to processor 211. Other implementations, such as application specific integrated circuits (ASIC) or neuromorphic architectures are equally useable.

[0080] There may be a number of different ways to invoke the machine learning model, which all fall within the meaning of “evaluating the (trained) machine learning model” as recited throughout this disclosure. For example, evaluating the model may involve calling an API routine to send the context input to a server and the server then performs the calculations according to the trained machine learning model and returns the results. In other examples, evaluating may involve issuing a command to local hardware, such as a local chip, device, machine learning accelerator (e.g., a USB device design to efficiently perform machine learning tasks or NVIDIA’ s Deep Learning Accelerator (DLA)), etc., that has the trained machine learning model stored thereon and provides a command interface to interact with the model. It is also possible to have a local copy of the machine learning model available so that the calculations are performed by the main processor of the local machine. Other local, remote or distributed implementations (such as cloud computing environments) are equally useable.

[0081] In some embodiments, the machine learning models may be trained on a broad range of different data such that it can be applied across a wide range of use cases. Such a machinelearning model may be referred to as a “foundational model”. Some foundational models that are applicable to the disclosed method include those that are publicly available and / or trained on publicly available data. In other embodiments, the machine learning model may be trained on a specific set of training data, in order to focus the generated outputs of the machine learning model to a specific task or area of interest. In yet other examples, the foundational model is further trained on the specific set of training data to improve the model in the area of interest.

[0082] In some embodiments, any one of the machine learning models may be a multimodal machine learning model, in which multiple inputs of different modalities (e.g., text, image data and audio data) are used to provide one or more generated outputs. An example of a multimodal machine learning model is an object detection model, which detects the location of a specific object (specified by input text, for example) in an image. This example model may generate output text that describes the location of the specified object in the image. Although the multimodal machine learning model can be evaluated on multiple input of different modalities, the multimodal machine learning model can also be evaluated on a single input and still generate an output based on the single input.

[0083] The machine learning models may be trained to generate output text based on input text and hence, may be a chat-based machine learning model. The machine learning models may also be referred to as a trained generative language model. Both the input and output text may be in the form of “natural language” (i.e., any language that occurs naturally in a human community by a process of use, such as spoken English, for example). Such machine learning models may be referred to as “chatbots”. A chatbot (which may also be referred to as a chatterbot) is designed to mimic human conversation (using natural language) through text or voice interactions. More particularly, the chatbot responds to input natural language using output natural language.Examples of such chatbots currently include ChatGPT (using GPT-3 or GPT-4), Microsoft’s Bing Chat (which uses OpenAI's GPT-4) and Google’s Bard.

[0084] In more specific examples, the machine learning models may be a large language model (LLM). A LLM is a type of artificial intelligence system, characterised by its massive training data and high volume hyperparameters. These language models ingested input text sourced from various sources and use fine-tuning to predict potential tokens or words. This enables them to perform various natural language processing tasks, including sentiment analysis, document classification, and lexical analysis, among others. However, their capabilities have extended beyond these tasks to encompass a broader range of applications and industries, such as chatbots,content generation, and even scientific research, demonstrating their versatility and growing significance in the field of Al.

[0085] LLMs like GPT (i.e., GPT-3, GPT-3.5 and GPT-4) and LLaMA are built upon the transformer architecture. These models are pre-trained on vast volumes of text data collected from publicly available online resources, and have been specifically trained to understand and process natural language. However, they differ from traditional natural language processing (NLP) systems in how they handle NLP processing tasks. Instead of relying solely on pretraining and fine-tuning, LLMs excel in NLP tasks through the technique of prompt engineering. In prompt engineering, tasks are conveyed as text descriptions, and these descriptions are presented to the model for interpretation and the generation of corresponding responses.

[0086] The LLM may be an artificial neural network, such as transformer model (e.g., a generative pre-trained transformer) which utilises encoder and decoder networks. These LLM are (pre-) trained using self-supervised learning and semi-supervised learning. In essence, LLMs are trained to predict what word comes next in a sequence of words which can be based on the semantic closeness of the words. LLMs form these predictions by ‘tokenising’ the words in the output text (i.e., converting the words into a vector of numbers). These tokens can also incorporate other information such as the position of the word in the input text and the information about the adjust words.

[0087] The process of creating these tokens is also referred to as ‘encoding’ or ‘semantic encoding’, as information regarding each word is essential encoded into a vector of numbers. The process of creating these token that incorporate the position of each word in the input text is referred to as “positioning embedding”. The encoding process also includes embedding at least one learnable parameter, which is determined through training. The opposite process then occurs where the tokens are converted from a vector of numbers into words, which is referred to as ‘decoding’. This gives the final output of the LLM in the form of text.

[0088] The machine learning model may also comprise an attention mechanism that applies weights to the tokens and may be characterized by its self-attention layers. These layers enable the model to assess the significance of words in an input relative to each other, thereby providing a more nuanced comprehension of the text. The attention mechanism may also include a scaled dot-product between different matrices generated by the model to calculate the weights. In simple terms, the attention mechanism allows a model to focus on different parts of the inputwhen generating each element of the output. This dynamic focusing capability results in a more contextually aware model, producing better results in tasks like translation, summarization, or text generation. Other ways of achieving attention within the model would be equally possible.

[0089] As a result, the machine learning model can process a large number of input values, such as an input text paragraph, at one time rather than sequentially in order to consider the context of each word. Nevertheless, the overall number of parameters in the machine learning model is relatively large, which is the reason those models are referred to as large models, such as LLMs or Large Action Models (LAMs). In some examples, a model is large if it has more than 100 million parameters or more than 1 billion parameters or more than 1 trillion parameters.

[0090] In some embodiments, the machine learning models referred to in this disclosure may be multiple machine learning models that have a ‘global’ input and a ‘global’ output. In that sense, the multiple machine learning models may be “daisy chained” together, such that the output of one machine learning model becomes the input for the next machine learning model within the chain. Some of the multiple machine learning models may operate in parallel, rather than in a series or chain. Each of the multiple machine learning models may have its own memory and / or have access to a common memory that is shared become some or all of the multiple machine learning models. In this sense, the multiple machine learning models may resemble parallel processing or parallel computer, such a computer architecture with multiple CPU cores which can be operated in parallel.

[0091] In some embodiments, the subset of the assembly language instructions comprises a first subset, each of the first subset comprising an operand corresponding to at least one of the one or more input parameters. In assembly language instructions, an operand may refer to a part of an instruction that specifies the data to be operated on or the address of the data. For example, the instruction “push eax, [ebp+phkResult]” has two operands. The first operand is “eax” and the second operand is “[ebp+phkResult]”. In one example, processor 211 may determine that “eax” is an input parameter or indicative thereof. Hence, the first subset may comprise assembly language instructions that comprise an operand with “eax”. As such, each of the first subset of assembly language instructions are linked to the one or more parameters as they comprise an input parameter as an operand.

[0092] In some embodiments, the subset of the assembly language instructions comprises a second subset, each of the second subset being linked to one or more of the first subset. Thesecond subset may comprise assembly language instructions that are not directly linked to the one or more parameters, but rather are indirectly linked to the one or more input parameters. For example, an instruction in the first subset may be linked to an instruction in the second subset as it is the same type of instruction e.g., a push and mov instruction. In another example, an instruction in the first subset may be linked to an instruction in the second subset as it is the similar type of instruction. Processor 211 may determine the names of the assembly language instructions (from a disassembler, for example) and may then determine that an instruction in the first subset is linked to an instruction in the second subset as the names of the instructions are semantically related or similar.

[0093] In some embodiments, each of the second subset comprises an operand corresponding to a shared parameter, wherein at least one of the assembly language instructions preceding the function call instruction comprises the shared parameter and at least one of the one or more input parameters. For example, processor 211 may determine that “eax” is an input parameter or indicative thereof. Processor 211 may also determine that the instruction “mov ecx, eax” is linked to the input parameter as it comprises the input parameter as an operand. Hence, the instruction “mov ecx, eax” may be part of the first subset. Processor 211 may then find the instruction “add ecx, OCh”, which does not comprise the input parameter “eax”. However, this instruction is linked to the instruction “mov ecx, eax” as each instruction comprises the parameter “ecx” as an operand. Hence, “ecx” may be referred to as a shared parameter.Therefore, the instruction “add ecx, OCh” is indirectly linked to the one or more parameters and used in the context input. As such, the instruction “add ecx, OCh” may be part of the second subset.

[0094] In some embodiments, the assembly language instructions define an instruction order, and processor 211 may determine 302 the subset of the assembly language instructions by considering the assembly language instructions in reverse instruction order. For example, the instruction order may be based on the order in which a computer executes the instruction in a compiled binary file. In other words, the instruction order may be indicative of an execution order. Hence, processor 211 may determine 302 the subset of the assembly language instructions by considering the reverse of the execution ordering. For example, processor 211 may first identify 301 the function call instruction then processor 211 may determine 302 the subset by working backwards from the function call instruction. This may be referred to as backtracking. In this sense, the assembly language instructions can be considered as an ordered list or an ordered array and the subset is then a sub-list or a sub-array.

[0095] In some embodiments, the context input comprises parameter name strings of the one or more input parameters. For example, processor 211 may determine the parameter name strings by receiving data from server 220 or from a disassembler. It is noted that the parameter name may be different from the parameter string in the assembly language instructions. For example, an assembly language instruction corresponding to one of the input parameters may be “push esi”. However, the parameter name is “len”, which may not be directly determined from the assembly language instructions. Having the parameter name in the context input may provide additional contextual information which may increase the accuracy of the predicted function identity.

[0096] In some embodiments, processor 211 creates 304 the context input by creating a symbolic string for one or more of the subset of the assembly language instructions. Processor 211 may then use the symbolic string in the context input rather than explicitly using assembly language instruction. For example, for the instruction “mov eax, [ebp+var_8]”, processor 211 may create a symbolic string for the second operand of the instruction. As such, processor creates the following symbolic string of the assembly language instruction “mov eax, mem”, as the operand “[ebp+var_8]” corresponds to a memory address. This may be referred to as symbolic mapping. Creating a symbolic string may be based on predetermined rules and / or a look-up table.

[0097] In some embodiments, processor 211 creates a symbolic string for one or more of the subset of the assembly language instructions corresponding to one or more of: a memory address; a pointer address; a complex expression; a hexadecimal value representing a location with the assembly language instructions; and an external function. Creating a symbolic string helps to reduce out-of-vocabulary (OOV) problems and enhances the generalisation capabilities of the trained machine learning model. The symbolic string may comprise the same amount of contextual information as the actual operand string. Hence, creating the symbolic string prevents the machine learning model from learning the actual operand or recognising patterns in the actual operand. This may also reduce the training effort of the machine learning model, and increases the accuracy of the trained machine learning model.

[0098] In some embodiments, processor 211 may receive a complied software file and apply a disassembler to the complied software file to generate the assembly language instructions. For example, server 220 may store the compiled software file (which may be a binary) and transmit the compiled software file to processor 211, which then applies a disassembler to generate theassembly language instructions corresponding to the compiled software file. The disassembler may be Interactive Disassembler (IDA), for example. The disassembler may be considered to be a decompiler, for example.

[0099] In some embodiments, the trained machine learning model is a language model. A language model in machine learning is a type of model designed to understand, generate, and manipulate human language. These models are trained on large datasets of text and learn to predict the probability of a sequence of words. The trained machine learning model may also be a masked model. A masked model uses masking techniques to process input data. For example, parts of the input may be blurred or missing. During training, such masking techniques may be used such that the machine learning model learns to recover the missing parts of the input.

[0100] In some embodiments, the trained machine learning model is at least partially trained by tokenising the context input. “Tokenising” in the present context refers to transforms raw text into a format that can be more easily analysed and processed by machine learning models.Processor 211 may tokenise the context input by converting each word in the context input into a number or a variable. “Tokenising” in the present context may have a similar meaning to embedding or encoding.

[0101] Further, in some embodiments, the trained machine learning model is at least partially trained by randomly masking one or more tokens of the tokenised context input. For example, the context input may be “push eax lea eax complex mov edx 3” and processor 211 may randomly mask one or more of the words in the context input, such as “eax”, “complex” or “mov”. As such, the machine learning model may be trained to recover the masked tokens. In other words, in some embodiments, the trained machine learning model may be trained to output the context input in its complete form (e.g., without masking).

[0102] In some embodiments, the context input comprises the function call instruction (or part thereof) and the trained machine learning model is at least partially trained by masking a token corresponding to the function call instruction. More specifically, the context input may comprise the function identity determined from the function call instruction, which may or may not correspond to the true function identity (e.g., the function identity in the function call instruction may be obfuscated). In other words, the trained machine learning model may be trained to recover the function identity using the contextual information in the rest of the context input.

[0103] In some embodiments, the trained machine learning model is at least partially trained by using training data comprising one or more of: a malicious function; and an obfuscated function. In the context of the present disclosure, “obfuscated function” and variations thereof may generally refer to a function defined within the assembly language instructions whose name has been purposely obscured. Obfuscated functions may evade detection be malware detection software, which may be useful tools for malicious entities and malware designers. In the context of the present disclosure, “malicious function” and variations thereof generally refers to a function defined within the assembly language instructions that may disrupt, damage, gain unauthorised access to computer systems, or the like.

[0104] In some examples, there may exist databases of existing assembly language instructions with at least one obfuscated function and the true identity of the function may be known. Further, in some embodiments, the trained machine learning model is fine-tuned after at least partial training by using training data comprising at least one stripped set of assembly language instructions. A stripped set of assembly language instructions may refer to a minimal or reduced set of instructions that are essential for a specific task or function. For example, the stripped set of assembly language instructions may only comprise the assembly language instructions, rather than any additional information such as debugging information or parameters names.Background, Threat Model, and challengesCalling of functions

[0105] Generally, function calling dictates exactly where a caller should place respective parameters that a function requires. It may use function parameters placed in specific CPU registers, on the program stack, or in both CPU registers and on the stack. For example, C calling convention specifies that the caller places parameters to a function on the stack in right-to-left order and that the caller (as opposed to the callee) removes the parameters from the stack after the called function completes, which can be seen in Fig. 4. Fig. 4 shows a real-world example of the API function call to sendto, demonstrating the API parameters push onto the stack (represented by the highlighted instructions). The comments after show the names of the parameters generated by a disassembler.

[0106] Fig. 4 illustrates an example of assembly instructions of a disassembled malware binary file. API sendto expecting six parameters to be pushed in right-to-left order. Aligned with Ccalling convention, the API parameters are either placed in CPU registers (e.g., lolen, to, ten, buf, s)' or pushed directly to the stack when parameters are pre-defined constant (e.g., flags). The names of the API parameters may be provided by a disassembler. Some disassemblers can extract the names of API parameters and other debug information from a database of known standard API function signatures and annotate them in the program being disassembled. For example, the input parameter names are represented after where the parameter values that are passed to the API are represented by the operands of push instructions, as shown in Fig. 4.Threat Model

[0107] The threat model of the disclosed method considers an attacker who employs techniques to obfuscate functions or remove debug information before compiling source code into binaries. Such tactics aim to obscure the code and thwart analysis by security analysts and anti-malware scanners. The obfuscation of functions and binary stripping create challenges in identifying the true behaviour and intent of the binaries. By examining the given binaries, the disclosed method aims to reveal the intended purpose of the code based on the use of obfuscated functions, without access to the original source code. The goal of the disclosed method is to identify and understand the malicious behaviours or hidden functions within the binaries despite the lack of source code transparency.Implementation challenges

[0108] Challenges in extracting function -relevant assembly instructions for accurate function identity prediction.

[0109] Lack of semantic information in disassembly

[0110] Extracting information related to a function, such as an API, from disassembled code can be challenging because the relevant assembly instructions are not clearly indicated in the disassembly. For example, the push instruction may be used for both passing parameters required for a function call and implementing general computation logic, making it challenging to determine the intended effect. As shown in Fig. 1, CreateFileW takes seven input parameters (indicated by the comment) but there are nine push instructions in the standard API call, indicating that the remaining two instructions do not correspond to API parameters. To addressthis, the disclosed method provides a heuristic-based approach to accurately extract function calls and corresponding parameters from the disassembled code.

[0111] Context modelling

[0112] Once the function names and their parameters have been identified from the disassembled code, context around the functions should be created, as the values of the parameters alone are not sufficient to represent a function accurately. Using the values of the parameters solely results in low accuracy when predicting function identities. This is further complicated by the difficulty of extracting linked or semantically-related assembly instructions from disassembled code to represent an API function. To address this, the disclosed method provides an approach based on program analysis that uses semantic-based backtracking technique to extract linked or semantically-related assembly instructions. Preferably, the disclosed method performs this by parsing the disassembled code in reverse order.

[0113] Limited data in stripped binaries

[0114] Typically, stripped binaries lack debug symbols information, which otherwise provides valuable hints for functions names prediction. The names of function parameters may be valuable for function identity prediction, but this useful semantic information is often stripped in binaries.Example embodiment

[0115] An example embodiment of the disclosed method is now presented. This example embodiment is directed towards for predicting the identity of an API called within assembly language instructions. In particular, the example embodiment is directed towards predicting the identity of an API in the Windows OS, given that Windows OS presents different issues related to function identification, such as the lack of associated source code and large number of available APIs for Windows OS. However, it is noted that various aspects discussed in relation to this example embodiment are equally applicable to other embodiments of the disclosed method. The disclosed method is not limited to the example embodiment presented herein.Overview of the example embodiment

[0116] Figs. 5a and 5b provide an overview of the example embodiment. More specifically, Figs. 5a and 5b provide an overview of context input generation and training of a machine learning model using the generated context input. At 510 of Fig. 5a, an input binary is disassembled using Interactive Disassembler (IDA, https: / / hex-rays.com / ida-pro / ) to obtain assembly language instructions representing the binary. In other words, processor 211 receives a complied software file and applies a disassembler to the complied software file to generate the assembly language instructions. While IDA and variations thereof are used in the example embodiment, other disassemblers may also be used in other embodiments. For example, one disassembler that is used in the example embodiment is IDA-FLIRT (a variation of IDA).

[0117] At 520, processor 211 constructs context, by: identifying 521 API callsites (e.g., function call instructions) and input parameters from disassembly, backtracking 522, starting from parameters, in the disassembly to extract linked assembly instructions (e.g., semantically-related assembly instructions) for each API, resulting in API codeprints (i.e., the context input), and performing 523 symbolic mapping and normalization of API codeprints to form a sequence of words to input to the training step.

[0118] More specifically, at 520, processor 211 analyses the disassembled source code to construct API codeprints. An API codeprint may include (1) the API name provided by the assembly language instructions, (2) the names and values of its input parameters, and (3) a set of assembly instructions that are linked (e.g., semantically-related, in the sense that the instruction utilises the same registers) corresponding to each input parameter value.

[0119] At 530 of Fig. 5b, processor 211 may pre-train a machine learning model with the masked language model task using the generated API codeprints to perform automatic API deobfuscation for unstripped binaries. Processor 211 may fine-tune the at least partially trained machine learning model using stripped binaries. More specifically, the normalised API codeprints are used to train Google’s BERT model (https: / / research.goog! e / pubs / bert-pre-training-of-deep-bidirectional-transformers-for-language-understanding / ), which employs a selfsupervised learning technique to learn the contextual embeddings of the API codeprints. These embeddings are then used to predict the names of APIs. The outcome of the above three steps is a pre-trained machine learning model for API prediction that can be adjusted for diverse downstream tasks through fine-tuning.

[0120] More details regarding this example embodiment are described below.Pre-processing

[0121] In the pre-processing stage, processor 211 extracts assembly source code by disassembling the input binaries using IDA-Pro. Processor 211 extracts disassembled functions with corresponding assembly instructions. A goal of the disclosed method may be to analyse the low-level program instructions to reveal details about the implementation of obfuscated APIs, making it possible to understand their functioning and potentially develop solutions for deobfuscation. This is particularly relevant in the Windows OS scenario, as the original source code is unavailable. However, there may be other scenarios where the original source code is unavailable for other operating systems.API codeprints construction

[0122] To construct an API codeprint from the disassembled source code, three steps are followed: (1) processor 211 scans the code in the forward direction to locate the API call instruction (e.g., the function call instruction) and identifies the API name; (2) processor 211 extracts the values of its input parameters and use them to backtrack the code in a reverse direction and extracts the linked assembly instructions (e.g., semantically-related assembly instructions); and (3) processor 211 converts the API codeprints into a suitable format for machine learning training. Processor 211 then further processes the API codeprints in order to map specific assembly-generated operands, such as memory or pointer addresses, to symbolic representations like mem and ptr. This may help to reduce out-of-vocabulary (OOV) problems and may enhance the model’s generalisation capabilities. Algorithm 1 outlines the complete process of constructing an API codeprint, which includes the extraction of the API name in steps 1-9 and the extraction of the values and names of its parameters and their semantics in steps 10- 19.Algorithm 1 API codeprints constructionInput: Binary executable file fOutput: < codeprint}, ... , codeprintN>Disf) = Disassemble the input binary file using IDA-ProfnList = get list of all functions and respective source codes from Disf)form fn in fnList doE instrs = [ L]Jfor instruction inst in fn doIf not a ‘call’ instruction or ‘call’ to an external function thenadd inst to FinstisIf ‘call’ instruction and Windows API is called thenapi-name = get API name from instparams-list = []api-context = []for inst in reverse / ) do starts backtracking the source codeif ‘push’ instruction and annotation provided thenparamname= parameter name from IDA-FLIRTparamval= parameter value from instparamcon= Param eter-Value-B acktracker (paramal, Fivstis)add paramnameto params-listadd paramconto api-contextcodeprintn= [api-name, params -list, api-context

[0123] 1) API calls and parameters extraction

[0124] In practice, API calls are identified by searching for the operand of the call instructions in the disassembled code, e.g., “call RegDeleteKeyA” instruction in Fig. 6 at 610. To successfully execute a function, the necessary input parameters are supplied before calling the API (the push instructions in red-dashed rectangle in Fig. 6 at 610).

[0125] After extracting the API name from the function call instruction, processor 211 extracts the values and names of its input parameters. To achieve this, processor 211 iterates through instrs inreverse order to identify instructions that pass parameters to the API function. In other words, processor 211 creates 303 the subset of the assembly language instructions by considering the assembly language instructions in reverse instruction order. As previously described, parameters are pushed to the stack with a push instruction. Based on that, the following heuristic is used: if the instruction is push, it implies that the program is transferring parameters to the API function. To confirm this, processor 211 may check the annotations from IDA’s FLIRT functionality for the names of the parameters. If such annotations exist, the value and the name of the input parameter are stored in paramname, and paramval, respectively (Steps 12 - 15 of Algorithm 1).

[0126] Processor 211 extracts linked instructions (e.g., semantically-related instructions) by tracing back through the assembly language instruction based on extracted parameter values (step 15) in an iterative manner. An API codeprint generated using this procedure may consist of the API name (obtained from step 9), its parameter names and values (obtained from step 17), and the context for parameters (obtained from step 18).

[0127] 2) Semantic backtracking

[0128] Instead of solely relying on parameter values for API function name prediction, processor 211 begins with the parameter values and traces back through F^ to extract linked instructions (e.g., semantically-related assembly instructions). As a demonstration, consider Fig.6, which shows how processor 211 may trace back through 7?^ and extract instructions linked to RegDeleteKeyA. It is noted that RegDeleteKeyA takes two input parameters hKey and IpSubKey. The intuition here is if the parameter’s value is stored in a CPU register, it may be expected that instructions using that CPU register share the same context. Hence, processor 211 identifies and extracts instructions that perform operations using common registers into groups of semantically related instructions. As shown in Fig. 6 at 610, the first parameter for RegDeleteKeyA is pushed onto the stack using the eax register (highlighted in grey). Starting with this parameter value, processor 211 traces back in the code to extract semantically related instructions, as shown in Fig. 6 at 620.

[0129] Algorithm 2 outlines the backtracking process based on a parameter’s value. It takes two inputs: the parameter value pctrcimval(obtained from step 16 of Algorithm 1) and Fisstis. Pconrefers to linked instructions and Cregsrefers to CPU registers found in backtracking, both initialized steps 1-2. If pctrcimvalis a CPU register (e.g., eax in Fig. 6), it adds inst to Peon and its corresponding CPU register to Cregs(steps 3 - 5). Each instruction in Finstisis checked. If it calls an external function (e.g., “call sub 403EBC” in Fig. 6 at 610), it’s added to Pcon(steps 7-8). External function calls are considered to provide additional context (such as semantics) as they link fragmented contexts, so they are added to Pcon.Algorithm 2 Parameter-Value-BacktrackerInput: paramval, F^sOutput: Pcon: set of semantically related instructions corresponding to the input parameter paramvalPcon= [] semantically related instructions of paramval.C = [] >CPU registers involved with paramval.If paramvalis a CPU register thenadd inst to Pconadd paramvalto Cregsfor inst in reverse( F^^ ) doIf ‘call’ instruction directed to an external function thenadd inst to PconIf not a ‘call’ instruction thenoperands = get operands from instfor operand in operands doIf operand in Cregsand inst not in Pconthenadd inst to PconIf operand is a CPU register and not in Cregsand inst in Pconthenadd op i erand toC?elseadd inst to Pcon

[0130] If inst is not a call instruction, its operands are extracted, analysed for other similarity to the input parameters (e.g., semantic relation) to other extracted instructions based on CPU register usage (steps 9-15). For example, the instruction “push eax, [ebp+phkResult] ” shown in Fig. 6 at 610 has two operands. The first operand, eax, is already in Cregs), but the instruction is not in Pconlist. Therefore, it is added to Pcon(steps 10-13). However, when processing the second operand, a new CPU register ebp is discovered. The newly discovered CPU register is added to Cregs(steps 14-15), expanding the search scope for hunting-related instructions. As more instructions are backtracked, the number of CPU registers in the Cregsmay increase. This is because the input parameter value (paramal) may pass through several CPU registers before it is input as a parameter to the API function. This example embodiment is flexible as it progressively expands the search scope with newly discovered CPU registers during the backtracking process rather than restricting the search to a single CPU register which could limit the context of API parameters. As shown in Fig. 6 at 610, a total of three unique registers (eax, ebp, and ecx represented in bold font) are involved in extracting correlated instructions.

[0131] The outcome of Algorithm 2 is the extracted linked instructions Pconfor parameter paramvalfrom Finstis. The search space for this backtracking may be limited to the set of instructions within a function, as represented by Fisstis. The API codeprint for the RegDeleteKeyA API is shown in Fig. 6 at 620. The obtained API codeprints are further processed before being input into a machine learning model.

[0132] 3) Post-processing API codeprints

[0133] The API codeprints obtained from Algorithm 1 are further processed before used for training purposes. To transform the API codeprints into a more suitable format, processor 211 performed symbolic mapping and normalization by replacing the raw assembly-level data with more meaningful normalized symbolic representations. This can help the model better understand the relationships and patterns within the data. Additionally, this assists in mitigating the out-of-vocabulary (OOV) problem by mapping OOV words to symbols already present in the vocabulary, which helps the model to generalise effectively.

[0134] It is noted that normalizing the instructions too coarsely, such as stripping immediate values, may result in the loss of significant contextual information. On the other hand, a finegrained normalization approach that is close to instruction disassembly may lead to an OOV problem, as it generates an enormous number of unseen instructions. This example embodiment strikes a balance by categorizing symbolic mapping into five main categories, providing a diverse representation from various aspects of disassembly. The full list of symbolic mappings is provided in the Appendix in Table 11.

[0135] To achieve this, memory locations are first represented using symbolic names based on the complexity of mathematical expressions used by compilers to compute the actual memory addresses of a variable. For example, as shown in Table 11, the expression “[esi+8]” for computing a memory address is mapped to “mem ” because it is based on a simple addition expression, adding two memory addresses together. In contrast, a more complex expression like “ [ebp+ lOh+var C] ”, which involves three operands including a register, a hexadecimal value, and a variable address, is mapped to the symbol “complex”. Expressions with three or more operands are generally represented in this way which may help the machine learning model better understand the relationships and patterns within the data.

[0136] Second, disassemblers, such as IDA, assign names to variables based on their location relative to the saved return address. Processor 211 maps these locations to symbolic representations based on the distance from a reference point, treating locations that are closer differently from those that are farther. The locations are typically expressed as hexadecimals. The mapping of these values to addresses depends on the specific value of the hexadecimal. For example, lower values like OCh are mapped to smaller addresses ( saddr). Hexadecimal values with two to four characters are mapped to ( maddr), and values with more than four charactersare mapped to ( laddr). The “dword ” prefixes, indicating a double word (4 bytes) value, and the “off ” prefixes, denoting offset values, are mapped differently, as indicated in the Table 11.

[0137] Processor 211 maps user-defined functions to symbolic names (e.g., “extrfun ”). It is noted that multiple functions may be mapped to the same symbolic name. For example, user-defined function names prefixed with “sub 40523 ” are mapped to “extrfun ” . However, numeric values that are input as API parameters are not changed because they can provide specific context information, such as input flags or sharing options. Keeping such numeric values unchanged may help in accurate API function prediction. Finally, processor 211 removes nonASCII characters and punctuation and converts all instructions to lowercase. The resulting set of API codeprints, as shown in Fig. 6 at 640, is fed to the machine learning model.Pre-training a masked language model

[0138] In the example embodiment, the machine learning model is based on Google’s BERT. To train the machine learning model, processor 211 randomly masks a certain percentage (15%) of API context tokens and then predict the corresponding masked tokens, as shown in Fig. 5b. In the example embodiment, a 12-layer Transformer with 768 hidden sizes and 12 attention heads is used. The intermediate size of the feed-forward networks is 3,072. The machine learning model is pre-trained on the training set, which contains more than 4.7 million API codeprints, as will be discussed in the evaluation section. It is noted that at the pre-training step, labels are not used as the masked API modelling is self-supervised learning. The input to the model is tokenized, and the length of each API codeprint is set to 512 tokens.

[0139] During the pre-training process, the model runs for approximately 28,000 steps (60 epochs) with a batch size of 2,000. The learning rate is set to 2e-5, and the cosine learning rate decay technique is used. In contrast to the default BERT model, which uses a vocabulary size of 30.522, the machine learning model used in this example embodiment uses a vocabulary size of 10.522.

[0140] A smaller vocabulary size is chosen for the following reasons: (1) a smaller vocabulary size will be more effective when working with assembly instructions related to API calls. This is because normalization and symbolic mapping has already been applied to the API codeprints, resulting in a smaller grammar, hence, the vocabulary size is set to 10,522. In general, the choice of vocabulary size is a trade-off between performance and model size, and the best choice willdepend on your specific use case. (2) A smaller vocabulary size can also make the pre-training process more efficient by reducing the amount of data that the model has to process. This can speed up the pre-training process and reduce the memory and computational resources required.

[0141] Overall, the 28,000 training steps took about one day using four NVIDIA RTX A6000 GPUs with 48GB of GPU VRAM each.Fine-tuning on downstream tasks

[0142] Fine-tuning may be used to adapt the pre-trained model to specific tasks. This enables the trained machine learning model to adjust to the nuances and characteristics of the new task, improving accuracy and performance compared to using a pre-trained model directly. Stripped binaries are compiled computer programs that have had debugging symbols and other unnecessary information removed in order to improve performance and protect against reverseengineering or modification.

[0143] However, attempting API function name prediction for stripped binaries without finetuning will make the task challenging, resulting in poor performance due to the absence of crucial information. An evaluation that the disclosed trained machine learning model applied on stripped binaries results in an accuracy of only 30%. Therefore, fine-tuning the trained machine learning model may be useful to enhance the performance, as it enables the model to learn the unique patterns and characteristics of stripped binaries.

[0144] Normal binaries contain both the names and values of parameters, while stripped binaries only contain the values. Fig. 7 shows an example of the FindResourceA API under normal and stripped binary cases. As can be seen, in the stripped binary case, the API names and corresponding parameter names are missing / stripped, while in the normal case, both the API parameter names (shown in red font), and the corresponding values are present. The disclosed machine learning model is fine-tuned using self-supervised learning techniques using a large dataset of stripped binaries.

[0145] Once the model has been fine-tuned on stripped dataset, the trained machine learning model is evaluated on stripped binaries. As will be discussed in the evaluation section, finetuning the machine learning model significantly improved its performance. Furthermore, the pre-training-then-fine-tuning paradigm may be applied for other API-related tasks, such as predicting the sequences of API calls.Dataset and Ground Truth ConstructionDataset collection

[0146] A dataset consisting of 11,098 Windows malware portable executable (PE) binaries was collected for evaluation of the disclosed method. Malware PE binaries are preferred for training as these binaries are likely to exhibit behaviours that are different from benign programs and more indicative of malicious activities. By doing so, the API call prediction model can better identify and extract APIs’ malicious contexts which is important for improved performance. Additionally, the presence of benign binaries in the training data could potentially dilute the model’s ability to recognize APIs used in malicious contexts.

[0147] The dataset consists solely of unpacked malware binaries, as packed malware must be unpacked to reveal its true behaviour during runtime. Using only unpacked malware binaries for training and evaluation helps to ensure that disclosed trained machine learning model is not biased towards packing-specific constructs in disassembly. The hashes for unpacked binaries were obtained from VirusShare (https: / / virusshare.com / ) and were sourced from VirusTotal (https: / / www.virustotal.com / ). The batch mode functionality of IDA was used to disassemble 11,098 raw binaries. By leveraging IDA-python for comprehensive code analysis, 4,744,969 API codeprints were exacted from 4,214 distinct Windows APIs, comprising a total of 10,468,434 parameters (see Table 1). To prepare for BERT -based masked language model pre-training, the API codeprints were pre-processed using NLTK (https: / / www.nltk.org / ) to remove special characters and punctuation marks.Item Count Malware samples 11,098 / / unique APIs used 4,214 / / APIs and corresponding contexts 4,744,969 / / number of API parameters in dataset 10,468,434 # of API codeprints used for training 3,368,519 (90%)# of API codeprints used for testing 336,852 (10%)Table 1: Dataset statistics.Ground truth (labelled) dataset

[0148] The construction of the ground truth involves a two-step process. Firstly, IDA FLIRT was utilised to obtain annotations for the given API codeprint. The obtained annotations may not explicitly contain the names and input parameters of an API in a structured format because FLIRT is a signature-based technology that generates annotations for disassembled code in the form of strings, symbols, API input parameter names, and more. Hence, the heuristics approach described herein was used to extract API names and their corresponding parameter names. The output consists of API codeprints, where each API codeprint contains the API name and a list of corresponding API parameters.

[0149] Each parameter includes the parameter name, parameter value, and contextually related assembly instructions, as detailed in the output of Algorithm 1. Lastly, the obtained API names and corresponding parameter names obtained from FLIRT annotations were validated using Microsoft’s official documentation (https: / / leam.microsoft.com / en-us / docs / ), which serves as the ground truth. Microsoft documentation contain exhaustive list of all native APIs and their respective input parameters, hence serve as a ground truth. This ground truth is established by analysing over Ilk binaries.

[0150] Stripped binaries

[0151] Stripped binaries neither contain API names, nor corresponding parameters’ names in their disassembled code, thereby making the prediction of API names more challenging. In this case, each API codeprint contains only a list of API parameters, where each parameter includes only a parameter value and corresponding set of contextually related assembly instructions. However, the API names are missing for stripped binaries. To create a ground truth dataset for stripped binaries, the API names and parameter names were removed from each API codeprint (see Fig. 7) in the ground truth dataset obtained from normal binaries. This process resulted in API codeprints that solely contain parameter values (without parameter names) and contextual disassembled code instructions. The stripped API codeprints using the fine-tuned trained machine learning model were tested and compared against the predicted API names and the ground truth API names (derived from the removal of actual API names).

[0152] Obfuscated binaries

[0153] The API names and corresponding parameters in obfuscated binaries are often missing and are typically represented as dummy words such as dword_, sub_, or register addresses,complicating the validation of the prediction results. An additional ground truth dataset was constructed, which comprised both obfuscated versions and their corresponding normal binaries.

[0154] A dynamic API resolution technique was used to obfuscate APIs. This approach crafts binaries in a manner where, rather than directly linking to Windows API functions during compilation, the required DLLs (such as kemel32.dll) are dynamically loaded at runtime using functions like LoadLibrary, and then function addresses are resolved through GetProcAddress. This involved writing source code in C / C++, which was then compiled to generate the corresponding binaries. Multiple APIs, including CreateFile, WriteFile, CreateProcess, and SendMessage, were obscured to generate both obfuscated and normal versions of these APIs. The normal version of these binaries served as a ground truth. This focus is to ensure that the invoked APIs remained undetected by IDA’s Import Address Table (IAT). Examples were generated in the x86 architecture, given that the majority of malware is designed for this platform.Pre-training and evaluation environments

[0155] Experiments were conducted on two machines: (1) The pre-training of the BERT-based masked language model task was performed on a 64-bit Ubuntu 18.04 GPU server equipped with a 64-core Intel Xenon 4U AMD 77022.0GHz CPU, 32 GB of memory, 4TB of SSD storage, and 4 NVIDIA RTX A600048GB VRAM graphics cards, and (2) the evaluation of disclosed trained machine learning model was conducted on a 64-bit Windows 10 desktop machine equipped with a 6-core CPU, 16 GB of memory, and 500GB of disk storage.Evaluation

[0156] The evaluation of the disclosed method will now be described. In particular, experiments were conducted using the dataset described above to evaluate the performance of the disclosed method. In this evaluation, the example embodiment described above was used.Evaluation method and metrics

[0157] “Transformer pipelines” are a convenient and efficient method for using models for inference. In the experiments described herein, the fdl-mask pipeline from the Transformers library (https: / / malapi.io / ) was used to perform masked language model inference. The pipelineworks by masking tokens in a sequence with a special masking token and then asking the model to fill in the missing tokens. In the experiments described herein, the API names were masked from API codeprints to invoke the model to predict API names. The prediction was considered correct if the predicted API name (w ) matched the ground truth API name ( n ), otherwise it was considered incorrect.

[0158] The two performance metrics were considered: (1) API prediction accuracy: this measures the proportion of correct predictions made by the disclosed method compared to the total number of predictions. (2) Confidence score: this is a new metric, which measures the consistency of the disclosed method in determining the number and names of input parameters for an API. The confidence score relies implicitly on the accuracy of detecting the number of input parameters and correctly identifying their names for an API.API parameters extraction accuracy

[0159] The confidence score described above was computed for identifying API names and their parameters by analysing disassembly. For example, consider the API CryptReleaseContext, which takes two input parameters: hProv and dwFlags. If this API is used 200 times in a given dataset, and the disclosed method correctly identifies the number (i.e., two) and names of the input parameters (i.e., hProv and dwFlags) 80 times, then it can be concluded that the disclosed method is 80% confident that CryptReleaseContext takes two parameters, and they are named hProv and dwFlags.

[0160] The disclosed method achieved an accuracy of 91.4% in correctly identifying the number of parameters for a total of 4,123 unique Windows APIs. Additionally, the disclosed method achieved an accuracy of 90.5% in correctly identifying the names of input parameters of APIs, indicating a high level of effectiveness in extracting this information from disassemblies.

[0161] Fig. 8a presents the distribution of confidence scores for the number of parameters in 4,123 Windows APIs. Fig. 8b presents the distribution of confidence scores for the names of parameters in 4,123 Windows APIs. The confidence scores, ranging from 0 to 1, are displayed on the x-axis in Figs. 8a and 8b, while the frequency or number of APIs with a given score is represented on the y-axis. Figs. 8a and 8b show that the majority of APIs have high confidence scores, as indicated by the longest bars on the right side of the graph. The mean confidence score for the parameters count is 0.91, with a standard deviation of 0.15 and a variance of 0.02.Similarly, the mean confidence score for correctly identifying the names of input parameters of an API was found to be 0.90, with a standard deviation of 0.16 and a variance of 0.02. These results were validated using Microsoft’s API documentation.

[0162] In summary, these evaluation results indicate that the disclosed method is highly effective at extracting the correct number (with an accuracy of 91.4%) and names (with an accuracy of 90.5%) of input parameters of an API from disassembled binary. This implies that the obtained API codeprints are trustworthy and can be used accurately trained the disclosed machine learning model for API predictions.API name prediction in normal binaries

[0163] The testing dataset, shown in Table 1, consists of 336,852 API codeprints. However, this dataset was filtered because it included APIs that take no parameters (e.g., GetProcessHeap, GetTickCount). As a result, a total of 272,334 API codeprints were used for evaluation of the disclosed method. Using this filter testing dataset, the disclosed method correctly predicted the names of 233,598 APIs, resulting in an overall accuracy of 85.77%. The disclosed method successfully predicted the actual names of 1,204 unique Windows APIs in the dataset, indicating that the disclosed method is not biased toward predicting a specific subset of APIs. The disclosed method is also able to consider 165 times more APIs compared to rule-based approaches, making it a practically deployable solution for predicting API names in the case where they are obfuscated, misleading, or resolved at runtime a given input binary.

[0164] The evaluation set included APIs with input parameter counts ranging from 1 to 14, with the breakdown of APIs by parameter count detailed in the Appendix in Table 10. Notably, approximately 84% of the APIs in this dataset have four or fewer input parameters. Despite this diversity in the input parameter counts, the overall performance of the disclosed method remains robust.

[0165] In summary, the disclosed method is highly effective in predicting the actual names of the obfuscated APIs, achieving an accuracy of 85.77% and correctly predicting the names of 1,204 distinct Windows APIs in the dataset described above, which included APIs with input parameter counts ranging from 1 to 14.API name prediction in the wild

[0166] To assess the generalizability of the disclosed method, an additional set of 1,148 malware binaries were obtained and a dataset of 995,422 API codeprints was constructed for evaluation. The previously constructed pre-trained model from the previous evaluation section was used without retraining the model. The disclosed method correctly predicted the names of 884,018 APIs, achieving an accuracy of over 88% over the previously unseen malware binaries. The number of distinct APIs successfully predicted in this dataset is 1,067.

[0167] The performance of the disclosed method with respect to the number of input parameters of the APIs was also evaluated, and the results of this evaluation are shown in Table 2. These results show that the prediction accuracy is consistently above 86% across all API input parameter counts. Additionally, it can be seen that the prediction accuracy generally increases as the number of API input parameters increases (for input parameter counts ranging from 1 to 6). Although the difference is not particularly significant, this may be because an increase in the number of input parameters also increases the amount of contextual information available, making it easier to accurately predict the API based on this context.# parameters Test samples Correct pred. Accuracy Unique APIs 1 449,466 387,035 86.11% 257 2 187,568 167,441 89.27% 244 3 126,297 116,397 92.16% 187 4 104,439 95,300 91.25% 161 5 48,945 46,576 95.16% 84 >6 78,707 71,269 90.55% 134995,422 884,018 88.80% 1,067Table 2: API prediction in the wild.

[0168] Additionally, in the Appendix, a macro-average analysis to assess the performance of the disclosed method across each API class is presented, ensuring that the results are not skewed by an imbalance in API distribution.API name prediction in stripped binaries

[0169] To evaluate the API prediction accuracy on stripped binaries, a new dataset is constructed by removing parameter names and other debug information from the disassembly, as depicted in Fig. 7. This dataset is then used to fine-tune the pre-trained machine learning model for the downstream task of API prediction on stripped binaries. After fine-tuning, a total of 991,561 API codeprints were tested. The disclosed method correctly predicted 821,895 APInames, resulting in an overall prediction accuracy of 82.88%. The performance is still reasonable, despite the testing on stripped binaries, demonstrating the effectiveness of the disclosed method. Table 3 summarises these results.# parameters Test samples Correct pred. Accuracy Unique APIs 1 449,472 337,283 56.94% 319 2 187,084 164,763 73.34% 284 3 126,301 111,781 88.50% 184 4 103,504 91,120 88.03% 148 5 48,105 42,511 88.37% 83 >6 77,092 74,437 91.84% 127991,561 821,895 82.88% 1,145Tab e 3: API prediction against stripped binaries.

[0170] It is observed that the accuracy increased as the number of input parameters increased. This is likely because more input parameters provide more context, making it easier to make accurate predictions. For APIs taking a single input parameter, the accuracy is around 57%, but with 6 or more input parameters, the accuracy is over 91%. This demonstrates that having more contextual information can significantly improve the accuracy of API predictions.

[0171] In summary, the disclosed method performs well overall, with a prediction accuracy of 82.88% against stripped binaries. Table 10 in the Appendix provides examples of APIs and their respective input parameters.API name prediction in obfuscated binaries

[0172] The performance of the disclosed method against obfuscated binaries was evaluated for two scenarios:1. Runtime API resolution. When binaries are obfuscated through runtime API loading to evade detection. This leaves IDA totally unaware of the APIs’ presence in the binary, resulting in a lack of information about the API in IDA’s import table and disassembly, respectively. When an API is obfuscated, the disassembled code is unable to identify the names of APIs due to obfuscation, the call instruction refers to either arbitrary words (prefixed with “dword ”, “sub ”), pointing to memory addresses (e.g., [esi + var AJ), or a CPU register that holds a reference to the API. To evaluate, the ground truth dataset previous discussed was used. The evaluation showed that the disclosed method was able to deobfuscate 50% of the obfuscated APIs. The API codeprints extracted fromdisassembly were precise, leading to their accurate obfuscation. It is noted that these APIs were undetected by IDA, with no information present in IDA’s import table and disassembly, demonstrating the effectiveness of the disclosed method.2. IDA failed resolving API names When IDA fails to resolve API names in the disassembly, leading to dummy names representing the APIs. A separate set of 2,918 binaries was collected and disassembled them using IDA. The following criteria was used: (1) if the operand to the call instruction is either a dword memory addresses (e.g., [ esi+var A ) or a CPU register (e.g., esi, eax, etc.), and (2) if the input parameters are passed to the API using push or mov instructions. If these conditions were met, it was considered likely that the API was obfuscated. This resulted in 148,685 obfuscated API codeprints, which were included in the test set.

[0173] Table 4 summarises the dataset and evaluation results. A total of 148,685 samples were tested, which showed that the disclosed method correctly predicted around 47% of the obfuscated API names that IDA was unable to determine.Dataset Total malware binaries 2,918Test API codeprints (obfuscated) 148,685 Unique APIs successfully deobfuscated 65 Evaluation Correctly predicted API names 69,921API prediction accuracy 47.03% Table 4: Automatic API deobfuscation results.

[0174] In summary, the disclosed method is effective in not only deobfuscating APIs, but also determines the intent of malware based on the capabilities obtained from deobfuscated APIs. It is shown that the disclosed method correctly deobfuscated 69,921 APIs that the commercial disassembler IDA fail to identify.Comparisons

[0175] The performance of the disclosed method (more specifically, the disclosed trained machine learning model) was compared to other methods. For experimental evaluation, the same codeprints were provided to the disclosed method, as well as two other approaches to assembly code analysis via embeddings from masked language models (referred to as Method 1 and Method 2). Each method generates an embedding vector and a simple feedforward classifier of two hidden layers with ReLU activation is used for each model, compensating only for the input embedding size (768 for the disclosed method, for example) and train for 50 epochs. Thisevaluation was restricted to the 1024 most common API names in the dataset, which covers 96.93% of the training data and 98.98% of the test data. As shown in Table 5, the embeddings generated by the disclosed are superior for the task of API classification. The training data for this task consists of roughly 2.67 million codeprints (90:10 training / validation split) and 1.04 million codeprints for test evaluation. After 50 epochs, both Method 1 and Method 2 fail to exceed 70% accuracy in this test, whereas the disclosed method exceeds 95% within just a few epochs.Model (Emb. size) Validation Acc. Testing Acc.Method 1 (768) 61.97% 66.48%Method 2 (128) 55.75% 61.01%Disclosed method (768) 98.67% 98.99%Table 5: Classification on top 1024 APIs after 50 epochs.Robustness against adversarial attacks

[0176] The robustness of the disclosed method was assessed against different evasion techniques. These evasion techniques aim to modify binaries in a way that retains their functionality while evading static analysis-based malware detectors. For example, one technique employs optimisation algorithms to iteratively generate candidate transformations that can effectively fool machine learning based malware detection techniques.

[0177] This evaluation of this disclosed method includes assessing its performance against two categories of adversarial transformation types: In-place randomisation (IPR) and code displacement. In the IPR approach, functions and basic blocks in a disassembled binary are identified and four types of functionality-preserving transformations are applied to them, updating the binary with the modified assembly. Similarly, the code displacement method involves moving disassembled code to a new executable section to disrupt potential gadgets.Dataset preparation

[0178] 1,996 malware binaries were randomly selected from the original dataset and applied both IPR and code displacement transformations to create two separate sets of transformed binaries. These transformed binaries underwent the disclosed API codeprint extraction pipeline to form the test set for assessing the resilience of the disclosed method to adversarial manipulations. Table 6 shows the datasets containing 2,148 unique APIs, with 748,135 API codeprints extracted. The disclosed method accurately predicted the names of 667,001 APIs,resulting in a prediction accuracy of 89.03%. Additionally, the disclosed method correctly identified 1,126 unique APIs, indicating the strong performance of the disclosed method on a large set of unique APIs.# of # of # of # of # Type binaries unique Codeprints correct Acc. correct APIs Pred. APIs Original 1,996 2,148 748,135 667,001 89.03% 1,126 IPR-transformed 1,343 1,913 244,466 215,817 88.28% 1,134 Code 1,878 1,898 237,792 203,151 85.43% 1,144 displacementTable 6: API prediction against adversarial binaries.IPR transformation attacks

[0179] IPR-transformed binaries employ conservative binary randomization techniques, which include four types of transformations while preserving functionality. These transformations involve replacing instructions with equivalent ones of the same length (e.g., substituting ‘sub eax, 4’ with ‘add eax, -4 ’), reassigning registers within functions or sets of basic blocks (e.g., swapping all instances of ‘ebx ’ and ‘ecx ’), reordering instructions based on dependencies, and modifying the order of register values pushed to and popped from the stack to maintain consistency across function calls.

[0180] These adversarial manipulations affect several aspects used by the disclosed method for API codeprint generation. Alterations to instructions and register changes can directly impact the semantic backtracking process of the disclosed method for constructing API context. Similarly, instruction reordering and manipulation of p ‘ ush ’ and ‘pop instructions come into play when the disclosed method extracts API input parameters. These transformations serve as an effective means to assess the performance of the disclosed method against well-crafted adversarial binaries. This evaluation on IPR-transformed binaries achieved a high API prediction accuracy of 88.28%, successfully predicting the names of 215,817 out of a total of 244,466 test API codeprints.Code displacement attacks

[0181] Code displacement involves relocating code to a new memory section to prevent codereuse attacks. The original code, typically at least five bytes in size, is replaced with a jump (jmp ’) instruction, redirecting program control to the relocated code. For larger displaced code, any bytes after the j ‘mp ’ are substituted with ‘trap ’ instructions to terminate code blocks. An extra mp ’ instruction is added to steer program control back to the next instruction after the displaced code, resulting in transformed control flows of binaries. This serves very well to evaluate the resilience of the disclosed method against code displacement attacks using control flow manipulation. The disclosed method achieved 85.43% API prediction accuracy (cf. Table 6) while successfully predicting 1,144 unique APIs, demonstrating that the disclosed method is resilient against code displacement attacks that rely on changing the control flow of a program.Ablation Study and Model ExplanationAblation study

[0182] In the ablation study, the impact on the performance of the disclosed in predicting API names was investigated under the following three cases: (1) Full API codeprints: This includes parameter values, parameter names, and corresponding contextually related instructions for each input parameter. (2) Partial API codeprints: This includes parameter names and values, but not contextually related instructions. (3) Values-only API codeprints: This includes only parameter values. The test set corresponding to Table 1, filtered to removed APIs with zero input parameters, was used in this study.Test codeprints Parameter values only Param values and names Full API codeprints Model Fine-tuned (§ 3.5) Original (§3.4) Original (§3.4)991,561 19.35% 60.21% 88.80%Table 7: Ablation study with and without API codeprints.

[0183] Table 7 presents the performance of the disclosed method as the number of input parameters varies across three cases. It is demonstrated that the disclosed method achieves an average prediction accuracy of over 88% when full API codeprints are used. For the second case, the overall prediction accuracy is 60%.

[0184] However, performance significantly degrades for the third, reaching around 19%, when only parameter values are used for API name prediction. It is noted that in the third case, no additional information other than parameter values is provided. The results above show that API codeprints with contextual instructions is useful in accurate API name prediction.Explainability via qualitative evaluation

[0185] To further assess the predictions of the disclosed method, a qualitative study of the predictions made by the pre-trained machine learning model was conducted. Specifically, the cases where the disclosed made incorrect API predictions were investigated. To do this, the pretrained BERT model was used to compute the embeddings for both the predicted and the ground truth APIs for all incorrect predictions made by the model and then calculated the cosine similarity (CS) between the predicted and ground truth API names.

[0186] By analysing the incorrect predictions, it was found that the average cosine similarity between the predicted and ground truth API names was 0.72 in cases where the predicted API names did not match the ground truth. This suggests that, even in cases where the predictions are incorrect, the predicted and ground truth APIs are still connected in context to some degree. Based on the types of errors made in the predictions, these were grouped into four main categories. Examples of incorrect predictions in each category are predicted in Table 8. The second and third columns show the predicted and ground truth API names, respectively, and the fourth column indicates whether the input parameters of both (predicted and ground truth) APIs are identical or not. Finally, the fifth column displays the cosine similarity between the predicted and ground truth API names.Errors Predicted Ground truth Cosine category Parameters sim. (%) identical? loadimagea loadimagew Yes 0.9780 Encoding- Istrcpyw Istrcpyna Yes 0.9355 specific writeprivateprofilestringa writeprivateprofilestringw Yes 0.9924 gettextextentpointa gettextextentpointw Yes 0.9911 deletefilew createfilew No 0.7659 regdeletekeya cryptdestroykey No 0.8899 Domain- recv send Yes 0.8523 related socket loadimagew No 0.8943 charuppera charnexta Yes 0.8380 settextcolor setbkcolor No 0.9779 gettickcount drawtexta No 0.4816 Unalike send callnexthookex No 0.4218 anglearc scalewindowextex No 0.3478 hwnd shcreateitemfromparsingname NA 0.1907 nbar getmenu NA 0.7807 Non-APImaddr getprocaddress NA 0.6171##rgn ellipse NA 0.5214Table 8: Prediction errors and cosine similarity. The predicted and ground truth API names are presented in pre-processed form.

[0187] Encoding-specific prediction errors

[0188] In Microsoft Windows, some API names are distinguished by the letters ‘A’ or ‘W’ at the end. These letters indicate the character encoding used in the API. For example, the letter ‘A’ after the API name signifies that ASCII character encoding is used, while the letter ‘W’ indicates that wide character (unicode) encoding is used. As an example, the LoadlmageA API is used to load an icon, cursor, animated cursor, or bitmap using ASCII encoding, while LoadlmageW has the same functionality but with wide character encoding. Table 8 shows similar examples.

[0189] When the predicted and ground truth APIs differ only in character encoding, the disclosed method treats it as a wrong prediction according to the evaluation metric, even though both the predicted and ground truth APIs are actually the same, because the input parameters for these APIs and their functionalities are identical. Table 8 shows examples of encoding -related mispredictions. The cosine similarity between the predicted and ground truth API names is greater than 0.93, indicating that they are contextually related. This similarity is calculated by first obtaining the embeddings of the API names from the pre-trained model using Transformer Pipelines, and then calculating the cosine similarity between the resulting embedding vectors.

[0190] Domain-related prediction errors

[0191] In this case, the predicted and ground truth API names are different, meaning they may have different input parameters and functionality, but they may still be related in terms of the domain they belong to. As shown in Table 8, the predicted and ground truth APIs deletefilew and createfdew are related in terms of their domain, i.e., both the APIs are from the file system. Similarly, the send and recv APIs share the network domain.

[0192] In the graphics ( setextcolor, selbkcolor, charuppera, loadimagew ) and key management ( cryptdestroykey and regdeletekeyd) domains of the Microsoft Windows operating system, a few missed predictions were observed. The cosine similarity in this category was above average (ranging from 0.76 to 0.97), indicating that the APIs are contextually related due to their common high-level domain. However, as the APIs may be different, their input parameters may not necessarily be identical, as seen in Table 8.

[0193] Unalike prediction errors

[0194] If the predicted and ground truth APIs have no similarities in terms of name, functionality, parameters, and domain, they are considered dissimilar. This dissimilarity is reflected in the low cosine similarity values shown in Table 8. In these cases, there is no overlap between the predicted and ground truth APIs and they represent completely different concepts.

[0195] Non-API prediction errors

[0196] The category of non- API predictions refers to instances where the predicted APIs from the disclosed method are not actual APIs, but rather could be an input parameter (e.g., hwnd and nbar are parameters of graphics APIs), a mapped symbol (e.g., maddr from symbolic mapping, shown in Table 11), or a tokenized word (e.g., rgn). Since the model predictions are not valid APIs, the input parameter column is irrelevant for this category of errors.

[0197] Building upon these findings, prediction results may be enhanced by considering the contextual similarity of API names, as outlined in the Appendix.DiscussionFine-tuning

[0198] The disclosed method offers the potential for fine-tuning various downstream tasks. One such application is the prediction of pre-defined constant parameters, certain API parameters only take a small set of values. By using the context in API codeprints, this disclosed method can predict the values and names of these parameters. This capability could prove useful in uncovering the parameter values frequently used by malware authors to circumvent security measures. The disclosed method can also be fine-tuned based on the sequence of API calls in a binary, which is particularly important when the goal is to determine the behaviour of binary programs through analysing API calls.Comparison to a disassembler

[0199] While some disassemblers (such as IDA FLIRT) provide helpful annotations, they alone may be insufficient for accurately predicting API names. For example, stripped binaries lack annotations entirely, posing a significant challenge for analysis. The disclosed method relies on amodel fine-tuned for stripped binaries to predict API names.. Furthermore, IDA’s annotations cannot resolve API names for obfuscated binaries. This empirical analysis reveals that IDA failed to resolve API names for over 69,000 API calls. However, the disclosed method is capable of predicting API names based on partially provided information.

[0200] Moreover, annotations for parameter names from IDA alone cannot provide sufficient context information, which plays a vital role in predicting APIs, as demonstrated in § 8.1. For instance, parameters such as hFile, IpBuffer, and nNumberOfBytesToWrite are commonly found among API functions like BackUpWrite , WriteFileEx, and WriteFile. Approaches based on IDA annotation will struggle to perform well in such situations due to limited context issues. Finally, IDA offers instruction-level annotations, but determining which instruction passes a parameter to an API function and inferring API names from these annotations requires labour-intensive manual analysis by reverse engineers, resulting in hours of labour. In contrast, the disclosed method uses heuristics to extract precise parameter names not readily available from IDA.AppendixMacro-average analysis

[0201] Typically, there is often an imbalance in the distribution of APIs used in malware, and this can lead to performance evaluation biases. To evaluate the performance of the disclosed method against each API class, a macro-average analysis was conducted that computes prediction accuracy separately for each API class. Fig. 9 illustrates the distribution (logarithmic scale) of 3,241 unique APIs within the training dataset. The x-axis in Fig. 9 represents the APIs, while the y-axis denotes their respective frequencies in the training dataset. Evidently, the dataset exhibits an imbalance, with certain APIs occurring more frequently than others. The mean and standard deviation of the frequency for each API within the training set are 860 and 3,846, respectively.

[0202] To perform macro-average analysis, the test set corresponding to Table 1, filtered to removed APIs with zero input parameters, was used in this study. The test malware dataset contains a total of 1,364 unique APIs. Fig. 10 shows the histogram of API prediction accuracy. Prediction accuracy is determined by the correct predictions of a specific API name by the disclosed method divided by the number of times that API is tested. In the test dataset, each APIwas observed an average of 202 times, indicating a sufficient number of test instances. The prediction accuracy was plotted on the x-axis and the number of unique APIs that achieved the specified prediction accuracy on the y-axis. In the evaluation, it was observed that 1,042 unique APIs, accounting for « 76.3 % of all APIs, achieved prediction accuracies of more than 95% for correctly predicting API names. Among these, 706 APIs (51%) achieved a perfect 200% prediction accuracy. Conversely, 82 APIs demonstrated prediction accuracies below 1%, underscoring the challenge that the disclosed method faces in accurately predicting the names of these specific APIs.

[0203] To delve deeper into the factors contributing to the lower prediction accuracy of the 82 APIs, the frequency distribution of these APIs in the training dataset is illustrated as the leftmost bar in Fig. 10. This was compared with the frequencies of 1,134 other APIs, which are depicted by the two rightmost bars. It is observed that the average frequency of the 82 APIs (62) in the training dataset is 35 times less than the average frequencies of the 1,134 APIs (2,203), suggesting that the data imbalance within the training dataset impacts performance. Despite this imbalance in APIs within the training dataset, the disclosed method continues to perform well, achieving an average per-API-class prediction accuracy of 90.3%. The top 30 most frequent malware APIs, their frequencies in the training set, and test accuracies in the test set are detailed in Fig. 11. The most frequently used API is GetProcessAddress, which is employed in various malicious contexts, including code injection, API hooking, import table manipulation, antianalysis, and intercepting API calls for monitoring or modifying behaviour. Similarly, malware commonly uses the LoadLibraryA API to dynamically load DLLs for evasion purposes.Malware capabilities and intent

[0204] 65 obfuscated APIs were identified during the previous evaluation. These APIs were further analysed to understand the purpose of the malware and determine the functions or capabilities they provide. For example, if the malware calls APIs related to network communication or file manipulation, it could indicate that the malware is designed to communicate with other systems or manipulate files on the infected system.

[0205] A list of nine malicious intents that are commonly targeted by malware and can be achieved through the use of specific APIs was curated. The list of malicious intents, their descriptions, and the APIs associated with each intent were generated through manual effort and consultation of online resources. These intents include enumeration, injection, evasion, spying,network, anti-debugging, ransomware, dropper, and helper. A total of 442 APIs that are frequently used by malicious software was identified and these APIs categorized them into these nine intents.

[0206] From the 65 deobfuscated APIs, evidence of three malicious capabilities was found: (1) C2 communication, (2) spying, and (3) evasion. These capabilities were identified by comparing the 65 unique APIs to the curated list of 442 APIs associated with the nine malicious intents. The C2 communication APIs included accept, send, gethostbyname, socket, and ioctlsocket. The spying and evasion capabilities were represented by the APIs BitBlt, DrawTextExA and Select, respectively.Context-based predictions

[0207] One may consider the performance of the disclosed method when context-awareness is taken into evaluation criteria. More specifically, the prediction of the disclosed method is considered correct if it is contextually aligned with the ground truth, even if it is not an exact match. For instance, consider the use case where one is interested in determining the capabilities of an unknown binary executable based on the APIs used in the program. Under this scenario, if the predicted API name is send whereas the ground truth is recv, it might be appropriate to consider the prediction as “plausible” because both APIs exhibit network capability and are contextually-related (the cosine similarity between them is more than 0.85). It was observed that many predicted APIs have a contextual relationship with the ground truth even if the prediction is wrong, as discussed in domain-related API predictions in Table 8. Similarly, encoding-specific API predictions in Table 8 could be considered correct predictions under any use case because the API functionalities are precisely the same but are developed with different character encodings (i.e., ASCII and Unicode).

[0208] The context-aware prediction of the disclosed method was evaluated by relaxing the criteria for correct predictions, incorporating context into the evaluation criteria, and comparing its performance to traditional prediction standards. To achieve this, a new evaluation metric for context-aware predictions based on cosine similarity was used. First, embedding vectors for all APIs in the dataset using Transformer’s feature-extraction pipeline was evaluated. Then, pair-wise cosine similarities between APIs were computed. Finally, context-based groups for APIs were created, i.e., for each API, APIs from the dataset were selected having cosine similarity equal to or greater than 0.91. For example, the context group of the API send contains thefollowing network-related APIs (it is a subset of APIs): connect, recv, urldownloadtofilea, internetconnectw , ftpopenfdew. It was observed that a total of 2,537 groups were created because not all APIs will end up in groups based on the cosine similarity threshold.Evaluation Context-aware Actual predictions Improvement (%) predictionsNormal (§ 5.3) 88.80% 89.21% 0.41% Stripped (§ 5.4) 83.67% 82.08% 1.59% Obfuscated (§ 5.5) 55.38% 47.03% 8.35%Table 9: Context-aware API prediction results.

[0209] At the evaluation stage, the disclosed method assesses the accuracy of its context-aware API predictions by comparing them with the ground truth. If the predicted API differs from the ground truth, the disclosed method then determines if the ground truth API is included in the context group of the predicted API. In this case, the prediction is deemed to be correct, otherwise, it is considered incorrect. As highlighted in Table 9, it is observed that the prediction accuracy improved in all scenarios, but the greatest improvement was seen for obfuscated API calls, with an 8% increase compared to the normal (0.41%) and stripped (1.59%) binary cases. This improvement can be attributed to the fact the prediction accuracy in normal and stripped binary cases is already high, leaving little space for improvement, whereas this is not the case for obfuscated API calls.APIs and input parameters#of # of APIs Example APIs parameters1 812 GetProcessId, LoadLibraryA, GetStdHandle, GetDriveTypeA,SetCurrentDirectory 2 667 WinExec, SetFileAttributesA, GetComputerNameA, IsChild,InitializeSecurityDescriptor, CryptReleaseContext , ChangeDisplaySettingsA 3 606 OpenMutexA, WNetGetConnectionA, ioctlsocket,OpenlnputDesktop, NetScheduleJobAdd, WriteProfileStringA 4 407 NetUserAdd, HttpAddRequestHeadersW,CryptlmportPublicKeylnfo, CryptGenKey, NetUserGetlnfo,InternetCreateUrlA 5 247 WriteProcessMemory, WriteProcessMemory, InternetOpenA,NtQueryObject, ryptDeriveKey, CryptCreateHash 6 138 ShellExecuteA, SHSetValueA, LockFileEx, LogonUserA,WinHttpQueryHeaders, waveOutOpen, SymLoadModule7 102 CreateRemoteThread, RegSetValueW, WSAConnect, WSASend,WSARecv, RegGetValueA, CreateFileW 8 58 AccessCheck, HttpOpenRequestA, acm StreamOpen,CryptDecodeObjectEx, erlnstallFileA, egEnumKeyExA 9 34 NtWriteFile, RegCreateKeyExA, WSAIoctl,AcquireCredentialsHandleA, BitBlt, NtReadFile 10 14 CreateProcessA, CreateProcessW, RtlCreateUser Thread,DrawStateW, EnumServicesStatusExA, ZwMapViewOfSection 11 15 ChangeServiceConfigW, NtCreateFile, CreateProcessAsUserA,ZwCreateFile, CryptQuery Object, AlphaBlend 12 11 CreateWindowExA, RegQuerylnfoKeyA, RegQuerylnfoKeyW,SetDIBitsToDevice, InitializeSecurityContextA 13 6 CreateToolbarEx, Create ServiceW, Create Service A,DrawDibDraw, ICCompress, FCICreate14 2 CreateFontA, CreateFontW Table 10: The breakdown of APIs based on the input parameters in the test set corresponding to Table 1.Symbolic mapping and normalisationInput value type Original values Mapped to[esi+8] memMemory[ebp+lOh+var C] complexOCh saddrHexadecimal 0F6Ah maddr0FFFFFFF8h laddrunk unknown ptrRuntime resolution { offset, dword } ptroff runtime ptrFunctions sub extrfunNumeric 3 3 (unchanged)Table 11: Variable type, original code, and mapped code to normalize instructions.

[0210] It is noted that some examples provided herein relate to Windows OS and corresponding assembly instructions. It is noted, however, that the disclosed methods can be applied to any assembly instructions. This includes instructions from different compilers or different, such as GNU assembly, different syntax (including Intel and AT&T syntax) as well as different processor architectures, such as complex instruction set computer (CISC) or reduced instructions set computer (RISC), and includes the x86 instruction set with its variants. It is an advantage to train the machine learning model on the same type of assembly instructions as the later evaluation. However, some learning may be transferable between different compilers / assemblers and different architectures since instructions may have similar labels,which is an advantage because it may be unknown which compiler / assembler was used for a particular software.

[0211] It will be appreciated by persons skilled in the art that numerous variations and / or modifications may be made to the above-described embodiments, without departing from the broad general scope of the present disclosure. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

Claims

CLAIMS:

1. A method for predicting an identity of a function called within assembly language instructions, the method comprising:identifying, in the assembly language instructions, a function call instruction to call the function;determining one or more input parameters of the function based on the assembly language instructions preceding the function call instruction;creating a subset of the assembly language instructions comprising one or more of the assembly language instructions linked to the one or more input parameters;creating a context input for a trained machine learning model comprising the subset of the assembly language instructions; andevaluating the trained machine learning model on the context input to generate an output corresponding to a predicted identity of the function.

2. The method of claim 1, wherein the subset of the assembly language instructions comprises a first subset, each of the first subset comprising an operand corresponding to at least one of the one or more input parameters.

3. The method of claim 2, wherein the subset of the assembly language instructions comprises a second subset, each of the second subset being linked to one or more of the first subset.

4. The method of claim 3, wherein each of the second subset comprises an operand corresponding to a shared parameter, wherein at least one of the assembly language instructions preceding the function call instruction comprises the shared parameter and at least one of the one or more input parameters.

5. The method of any one of the preceding claims, wherein the assembly language instructions define an instruction order, and the method comprises determining the subset of the assembly language instructions by considering the assembly language instructions in reverse instruction order.

6. The method of any one of the preceding claims, wherein the context input comprises parameter name strings of the one or more input parameters.

7. The method of any one of the preceding claims, wherein the subset of the assembly language instructions further comprises one or more external functions called within the assembly language instructions.

8. The method of any one of the preceding claims, wherein creating the context input comprises creating a symbolic string for one or more of the subset of the assembly language instructions.

9. The method of claim 8, wherein creating the symbolic string comprises creating a symbolic string for one or more of the subset of the assembly language instructions corresponding to one or more of:a memory address;a pointer address;a complex expression;a hexadecimal value representing a location with the assembly language instructions; andan external function.

10. The method of any one of the preceding claims, wherein the method further comprises receiving a complied software file and applying a disassembler to the complied software file to generate the assembly language instructions.

11. The method of claim 10, wherein the complied software file is a binary file.

12. The method of any one of the preceding claims, wherein the trained machine learning model is at least partially trained by tokenising the context input and randomly masking one or more tokens of the tokenised context input.

13. The method of claim 12, wherein the context input comprises the function call instruction and the trained machine learning model is at least partially trained by masking a token corresponding to the function call instruction.

14. The method of any one of the preceding claims, wherein the trained machine learning model is at least partially trained by using training data comprising one or more of: a malicious function; and an obfuscated function.

15. The method of any one of the preceding claims, wherein the trained machine learning model is fine-tuned after at least partial training by using training data comprising at least one stripped set of assembly language instructions.

16. The method of any one of the preceding claims, wherein the trained machine learning model is trained to generate output text and evaluating the trained machine learning model on the context input generates an output text corresponding to a predicted identity of the function.

17. The method of any one of the preceding claims, wherein the one or more input parameters correspond to one or more parameters being pushed before the function call instruction.

18. The method of any one of the preceding claims, wherein the function is an application programming interface (API).

19. Software that, when executed by a computer, causes the computer to perform the method of any one of the preceding claims.

20. A system for predicting an identity of a function called within assembly language instructions, the system comprising one or more processors configured to perform the method of any one of claims 1 to 18.

Images