Enhancing security in the operation of a software application

Abstract

In general terms, the present disclosure proposes automatically processing a program module (e.g. a component of a software package) to identify vulnerabilities in it, and produce a modified program module in which those vulnerabilities are at least partially mitigated. The module vulnerabilities may include the use of one or more functions which are known to be vulnerable, and in the modified program module some or all of the calls to vulnerable functions may be replaced by calls to more trusted protected functions.

Description

[0001]Enhancing security in the operation of a software application Field of the invention The invention relates to a method of modifying a program module of a software application to enhance its security, such as to provide protections for the program module against string attacks or other security vulnerabilities. Background Modular programming is a programming technique in which an application is provided as one or more (typically multiple) independent program modules which execute one aspect of the functionality of the application as a sequence of commands. In the source code for a program module, the commands comprise a plurality of commands which are calls to functions (also known as procedures, methods or sub- routines), often selected from a library of defined functions provided by a software house. Each function takes an input of one or more arguments (parameters) associated with the function, such as numerical values or text, and generates a corresponding output. The format of the input sequence of arguments and output is referred to here as a type signature or simply “signature”. The arguments may be explicitly provided in the program module, or they may be provided as index values which reference positions in data structures where the values of arguments are stored. A programmer writing the application may be unaware of how any functions which reference a library operate in detail. It sometimes becomes desirable to modify a program module. This can happen for example, when a security vulnerability is identified in one of the functions. One known security vulnerability is a “string attack” in which a malicious actor (“attacker”) provides to a function an input string that has been designed to obtain a malicious result, such as extracting secret information, e.g. by referencing the location of secret information in the string in such a way as to cause the function to output the secret information. More generally, an attacker may exploit a security vulnerability to execute code, access private data, or cause a segmentation fault in the running application, or cause new behaviours that could compromise the security or the stability of the system. An example security vulnerability is the Log4j2 vulnerability (CVE-2021-44228). The malicious actor may for example be a user intending to run code on in a secure computing environment shared by many users, and who wishes to disrupt the operation of the secure computing 15027066-1 environment, or gain unauthorized access to secure information, such as information about the software the other users are running and / or the data it operates on and / or generates. A current method of dealing with an identified vulnerability involves reviewing the source code of each program module manually to identify instances of the vulnerable function, and modifying the source code manually so that the vulnerable function is no longer employed to obtain the same module functionality. This is a time-consuming process, and the module cannot be used safely until it is completed. Instances of the function are easily overlooked in this process. Furthermore is it is common for mistakes to be made in modifying the module, particularly if the operation of the module was not well-documented and / or if no-one familiar with the module is available to modify the source code. Furthermore, any data which has been obtained to demonstrate the reliability of the module becomes obsolete when the module is modified. In addition, where the modifications are made in response to a vulnerability discovered in one version of a library, these modifications may be incompatible with later versions of the library. Further modifications may then need to be made to ensure that the software can operate with later versions of the library (which may themselves have been updated to address the security vulnerability or other security vulnerabilities). Summary In general terms, the present disclosure proposes automatically processing source code for a program module (e.g. a component of a software package) to identify vulnerabilities in it (“module vulnerabilities”), and produce a program module in which those vulnerabilities are at least partially mitigated. The module vulnerabilities may include the use, in the program module, of one or more functions called in the source code which are known to be vulnerable. Brief Description Examples of the present disclosure are now explained with reference to the following drawings, for the sake of example only. Figure 1(a) is a schematic illustration of a processing device according to a comparative example. 15027066-1 Figure 1(b) shows schematically elements stored in a non-volatile memory of the processing device of Fig.1(a). Figure 2(a) is a schematic illustration of a processing device according to an embodiment of the present disclosure. Figure 2(b) shows schematically elements stored in a non-volatile memory of the processing device of Fig.2(a). Figure 3(a) is a schematic illustration of a first system comprising a processing device as shown in Fig.2(a); Figure 3(b) is a schematic illustration of a further system comprising a processing device as shown in Fig.2(a); Figure 4(a) is a schematic illustration of a further system comprising a processing device as shown in Fig.2(a); Figure 4(b) is a schematic illustration of a further system comprising a processing device as shown in Fig.2(a); Figure 4(c) is a schematic illustration showing an example hardware security module (HSM) device; Figure 5 is a flow-diagram of a method performed by an embodiment. Figure 6 is a flow-diagram of another method performed by an embodiment. Equivalent or near-equivalent items are given equivalent reference numbers in different ones of the figures. Detailed Description In the modular programming technique, applications are composed of multiple program modules written in one of a number of known programming languages. A program module may be subject to a number of vulnerabilities, which can be exploited by malicious parties. One class of vulnerability, referred to as code injection vulnerabilities, involves a malicious party providing input data that, when processed by a program module, causes the program module to behave in way that was not intended by the developers of the program module, such as by executing one or more commands specified by the malicious party. One example of a code injection vulnerability is a format string attack in which a malicious party provides an input string that contains a sequence 15027066-1 of characters that is interpreted as a format specifier (e.g. "%x.%x.%x.%x"), and that when processed by a format function (e.g. printf in the C programming language), is interpreted by the format function to cause undesirable behaviour, such as outputting data from or writing data to a location in memory. For example, the format specifier may define a number of additional input arguments that were not specified in the call to the format function, which may then cause the format function to access data that was not intended to be provided to users. Figure 1(a) shows a schematic illustration of a device 900 according to a comparative example. The device 900 comprises an input interface 901, a processor 905, working memory comprising RAM 911, an output interface 903, and non-volatile storage 907. Figure 1(b) shows a schematic illustration of various elements which are stored in the non-volatile storage 907. In this example, an application library is stored in the non-volatile storage 907. The application library comprises bundles of code that can be used repeatedly in different programs. Within the application library is stored an application package, which stores a Python script, Script 1. The Script 1 may correspond to an application. The Script 1 comprises computer program code – in this case Python code. When executed, the Script 1 is represented as a software product, or process, in the working memory. The application typically comprises a plurality of program modules. The processor 905 comprises logic circuitry that responds to and processes instructions in program code present in the working memory. Also stored in the non-volatile memory 907 are any components necessary to execute the Script 1, including in this example a built in language library, an interpreter and any third party libraries that are needed. When executed, a copy of the Script 1 is loaded into the working memory, together with any other components necessary to execute the Script 1. The built in language library comprises a set of functions {^^^^(^^^̅^)}^^^=^0 which are either part of the language or are part of the language’s internal or standard libraries. In this example the language is Python. As illustrated in Figure 1(b), the built in language library in this example comprises the following functions: ^ print() - prints to default output device ^ len() – returns length of object ^ abs() – returns the absolute value of a number ^ exec() – supports execution of code ^ format() - converts a value to a formatted representation 15027066-1^ ….Third party libraries, each containing a set of functions, are also stored in the memory 907. The Script 1 is a Python script which may call functions from the built in language library and / or the third party library. An interpreter is used to execute Script 1. In other examples, where a script is written in a different programming language, the device 900 may use a compiler instead of an interpreter to generate lower-level instructions for executing the program. A problem arises when one or more of the functions of the Built-in language library or a third party library is found to have one or more vulnerabilities. This will be illustrated with an example below. Script 1 is an example of a program which is vulnerable to a format string attack. An excerpt of code from Script 1 is shown below: SECRET_API_KEY = '8675309-3141592-6535897' class WebPage(object): def __init__(self, api_key): self.api_key = api_key def render(self, html_template, text): return html_template.format(text) def publish(html_to_publish): “”” Publish the webpage to a server using the API key.""" # Code for publishing omitted # receive html_template from user, e.g. “{text}” html_template = input(“Enter template:”) # receive text from user to insert into template text = input(“Enter text:”) #generate html for the web page from the text provided by the user html = WebPage(SECRET_API_KEY).render(html_template, text) # print the generated html print(html) In this simplified example, the Script 1 comprises code relating to generating and then publishing a web page. Implementations of the Script 1 may for example be hosted 15027066-1 on a computing platform that allows users to generate HTML code for a web page and then, if the user is suitably authenticated, publish the HTML code on a server. In this scenario, a user would not generally be able to read or edit the Script 1, since it is stored on the device 900 and is not accessible to users. To generate the HTML code, the user provides an HTML template string that comprises a field that acts as placeholder for user-supplied text that it is to be included in the HTML in the location of the placeholder. Publishing the HTML code may require the use of an API key (SECRET_API_KEY in the Script 1) that needs to be kept secret to prevent malicious users from being able to modify the web page (for example). When the user provides benign input, such as the template string “{text}” and the text “Hello World!”, then the Script 1 will generate, using the render method, a corresponding HTML string: “Hello World”, which is then printed to an output device, such as the user’s display device. However, if the user enters a maliciously crafted template string: “API key:{self.__init__.__globals__[SECRET_API_KEY]}” , together with, for example, the same text “Hello World!”, then the Script 1 will print the HTML string “'API key: 8675309-3141592-6535897'”, thereby revealing the secret API key. Thus, the malicious user can provide input to the device 900 that allows access to the global namespace (via __globals__) into which the SECRET_API_KEY is loaded when the Script 1 is imported or run. The present disclosure provides systems and methods that can be used to mitigate or avoid attacks such as this. In general terms, the present disclosure proposes a method, such as one performed by a protection layer of a computer system which provides an interface to a processing system, to increase the security of a program module. One specific expression of this concept is a computer-implemented method of modifying a program module including a plurality of program portions which each call a corresponding function. One of more of the functions belong to a pre-determined set of “pre-specified” functions, such as ones specified by a user (e.g. a system administrator, or user associated with the program module). The method comprises identifying functions called in source code for the program module which are one of the set of pre- specified functions. Each pre-specified function is associated with a corresponding replacement function (or “protected” function) for calling in place of the pre-specified 15027066-1 function. In some implementations, the replacement function can be chosen based on a definition file corresponding to the pre-specified function. The method further comprises processing (e.g. compiling) the source code to generate, for each call to a function in the source code for the program module, a corresponding program portion of the program module. For at least one of the identified functions, each of the generated program portions corresponding to the identified function calls the corresponding replacement function. The method also comprises outputting the program module, e.g. as (lower-level) instructions in a machine language or instruction set from the source code. For example, processing the source code to generate the program portions can comprise generating corresponding bytecode or machine (e.g. assembly) code for execution by a runtime environment or processor. In some implementations, the program module can be generated with minimal or no modifications to the source code. In implementations, the method can comprise processing the source code for the program module to generate a symbol table that comprises a respective entry for each of the functions called in the source code, each entry having a corresponding reference or pointer for calling the function, e.g. a memory address or offset, such as in working memory RAM 911 as mentioned above in connection with Figure 1(a), or a reference to a data structure or object that comprises information (e.g. a pointer) for calling the function. The method can then comprise replacing the reference or pointer in the entry for the at least one of the identified functions with a respective reference or pointer for calling the corresponding replacement function. The symbol table can then be used (e.g. by a compiler or interpreter) to generate the program module. For example, the program module can comprise instructions that include the reference or pointer for the replacement function rather than the reference or pointer for the at least one of the identified functions. Advantageously, by modifying the symbol table, the need to modify the source code can be avoided or minimised. Each of the functions that are one of the set of pre- defined functions can be identified using the symbol table, i.e. without having to search the source code for each pre-defined function, and the modification to the symbol table can guarantee that every call to an identified function can be automatically replaced by a call to the corresponding replacement function. It is not, for example, necessary for a developer to identify manually, e.g., by setting a flag or other identifier, that a particular function is used in the software before the replacement can be effected. Furthermore, 15027066-1 the replacement of the identified functions can be carried out dynamically, e.g., at a certain point in a program, for interpreted languages, such as python. In other words, the replacement can be carried out at run-time, rather than compile-time in some cases. The replacement of the identified functions can also be reversed such that the replacement function(s) can be called in place of the corresponding identified function(s) at one part of the program, and then the original (i.e., identified) function(s) can be called at a later part of the program (or vice versa), as set out in the source code. Generating a program portion corresponding to the identified function that calls the corresponding replacement function may be referred to performing a “replacement” operation. Similarly, a program module for which one or more replacement operations for identified functions have been performed can be termed a “modified” program module. Note that a replacement operation may not be performed for all the pre-specified functions. For some or all of the pre-specified functions, another operation may be performed on the corresponding identified functions to increase the security of the program module. In some implementations, a function known to have a vulnerability may be used as one of the pre-specified functions, and calls to the function in the source code of the program module may be removed by the present method from the program module, and replaced with one or more replacement functions (“protected functions”), e.g. chosen based on a definition file. Other uses of the present method are possible. For example, even if a certain function called in the source code of the program module is not presently known to be vulnerable, it may be a function from a library of functions (e.g. stored in the computer system which performs the method, or elsewhere) which is not trusted. If an entity performing the method distrusts this library, or has another reason for not wanting to use it, then the entity can perform the method to specify the functions of the library as the set of pre-determined functions. The entity may specify, (e.g. using an associated definition file or files), corresponding functions of a function library which the entity does trust or prefers, e.g. a library stored on the same computer system which performs the method. Thus, when the method is performed, calls to identified functions from the distrusted library are replaced by calls to protected functions which call the trusted library. The method may be conducted fully automatically, that is substantially without human involvement, except in some implementations by initiating the method. Because it is automatic, all instances of the pre-specified functions are expected to be found, e.g. 15027066-1 as a result of modifying the symbol table as described above. The modification of the program module can be conducted in a sophisticated manner, e.g. based on a definition file or files, which has been tested to reduce the risk of causing the (modified) program module to perform improperly or introduce new security vulnerabilities. As discussed in more detail below, the method may be carried out by a security- enhancement software application which is used to modify the program module at a time when the source code is being processed, e.g. the software application may be supplied to an author or developer of source code who can use it to improve the security of the program module. For example, it can be used as a protection layer prior to running the program module, e.g. during a compilation process or interpretation process. The interpreter / compiler may be configured to recognise an initiation command (e.g. triggered by an import statement) within the program module, and to perform the method upon recognizing the initiation command. Optionally, the method can be used by an interface to a secure computing system (e.g. an HSM) which is to run the program module, to provide a protection layer for the secure system. The interface may implement the interpreter / compiler. Optionally, the interface and / or secure computing system may continue to use the protection layer during the operation of the modified program module, as explained below. As noted, a definition file for a given one of the pre-specified functions may specify at least one different “protected” (replacement) function which is trusted (at least to a greater extent than the pre-specified function; for example, it may be a function from a trusted function library) and which is capable of the same data processing operations as the corresponding pre-specified function when that pre-specified function is operating as intended. In the case that there are multiple protected functions to replace a certain one of the pre-specified functions, the multiple protected functions are collectively capable of the same data processing operations as the corresponding pre-specified function when that pre-specified function is operating as intended. In some implementations, the arguments of the replacement function may comprise the arguments of the corresponding pre-specified function. For example, the replacement function may have the same arguments as the corresponding pre-specified function, or it may have other additional arguments as well. In another example, the replacement function may be used to replace a pre- specified function of the program module with a protected (replacement) function having additional security (i.e. even if no vulnerability has been identified in the pre-specified 15027066-1 function). For example, the pre-specified function and protected function may define respective cryptographic operations, where the latter is more secure than the former. For example, some security functions include a security operation which can be performed recursively (e.g. a hashing operation); that is, the output of a first performance of an encryption operation may be used as the input for another performance of the encryption operation, and this recursive process may be repeated one or more times. A pre- specified function which performs a security operation on its arguments a first number of times (e.g. only once, or recursively more than once) may be replaced by a protected function in which the security operation is performed recursively a second number of times greater than the first number of times (i.e. the cardinality of the recursion is increased). For example, a pre-specified function which performs a hashing operation (or other encryption operation) once may be replaced by a pre-specified function which performs a hashing operation multiple times recursively (i.e. each hashing operation except the first is performed on the output generated by a preceding one of the hashing operations). In some instances, the replacement operation for a given identified pre-specified function is carried out subject to certain conditions, e.g. only if it is determined that a “vulnerability criterion” is met. For example, a certain function may be safe for certain choices of its arguments and not for others. In an implementation, the pre-specified function may only be replaced if the argument(s) meet a vulnerability criterion, e.g. the instances of the arguments would comprise, or make it uncertain that the function can operate without compromising, the security of the secure computing environment. The concept of providing a vulnerability criterion to enhance the security of the program module is not limited to the method discussed above of modifying the program module. In some situations, a check of whether the vulnerability criterion is met may be performed when the modified program module is run, e.g. as a protective layer of a secure computing system. For example, in some implementations, the replacement function can call the identified function (or at least a copy of the identified function). That is, the replacement function can act as a “wrapper” function for the identified function. In such cases, the replacement function can, for example, monitor one or more of: the data provided as input to the identified function; the data output by the identified function; computational resources accessed by the identified function; and the execution time for the identified function. The replacement function can additionally or alternatively handle any errors that occur when the identified function is called, e.g. by handling or catching exceptions or errors raised during execution of the identified function. 15027066-1 In one example, the vulnerability criterion may be whether the argument(s) of each identified function references one of a (pre-determined) set of resources. Upon determining that the argument(s) of one of the identified functions references one of the set of resources, one or more actions can be taken to prevent the one of the set of resources being accessed. For example, a resource other that the one of the set of resources, such as a non-sensitive or public resource, can be provided in place of the one of the set of resources. When the replacement function acts as a wrapper function for the identified function, one or more operations of the replacement function can evaluate the vulnerability criterion at run time. If the vulnerability criterion is met, a corrective action may be taken. For example, the corrective action may comprise suspending or terminating the execution of the program module (or the application as a whole), and / or transmitting an alert to a system operator to warn the operator of a need to perform a manual corrective action (i.e. to issue a command to the computer system), e.g. to prevent, suspend or terminate the program module (or the application as whole). The corrective action can comprise logging the identified vulnerability criterion and / or identified function. Alternatively, the operation of the program module may be allowed to continue after the determination of that the vulnerability criterion is met, but e.g. using modified arguments in the input to the pre-specified function, where the arguments are selected as arguments which do not meet the vulnerability criterion. For example, if the vulnerability criterion comprises whether the input arguments of each identified function references one of a set of resources, the corrective action may comprise modifying the input arguments so as not to reference any of the set of resources. In another example, the determination of whether the vulnerability criterion is met may comprise comparing outputs generated by an identified function with a database of secure information, to identify matches between the outputs of an identified function. Upon identifying a match (which may be indicative of the function having gained unauthorized access to the secure information), the wrapper may cause a corrective action. For example, if the replacement function (or, more generally, the protection layer) identifies that an identified function has managed to extract some of the secure information, it may terminate or suspend operation of the program module (or an application of which it is a part), and / or generate an alarm to a system operator, and / or modify the data output by the identified function to remove the secure information. 15027066-1 In another example, the determination of whether the vulnerability criterion is met may include monitoring a time taken by an identified function to run, and, upon the run time excessing a threshold, taking a corrective action (e.g. suspending / terminating the modified program module and / or generating an alert to a system administrator. The concepts presently disclosed may be expressed as a method, or alternatively as a computer program product (e.g. program instructions stored on a tangible recording medium, or transmitted as a signal over a communications network), and operative to cause one or more processors (e.g. in spaced apart locations) to perform one of the methods. For example, the computer program product may be a product supplied to an author of a software application, such that the author can use the computer product to enhance the security of the application. The concepts may also be expressed as a computer system comprising one or more processors (e.g. multiple processors, optionally in spaced apart respective locations), and a data storage device storing program instructions which, when implanted by the one or more processors cause the one or more processors to perform the method described above. Figure 2(a) shows a schematic illustration of a device 900 according to an embodiment. The device 900 comprises an input 901, a processor 905, working memory comprising RAM 911, an output 903, and non-volatile storage 907. These components will be described in further detail below. Figure 2(b) shows a schematic illustration of various elements which are stored in the non-volatile storage 907. In addition to the application package, the application library also comprises a protection package, denoted “protection-layer” in this example, which is itself an embodiment of the present invention. This package is configured to protect against a format string attack, such as that described with reference to Figure 1(b). The protection package comprises a file comprising code that can be used repeatedly in different programs. The Script 2 is a software application containing program modules which can be modified. Script 2 comprises computer program code – in this case Python code - stored in the non-volatile memory 907. When executed, the Script 2 is represented as a software product, or process, in the working memory. The processor 905 comprises logic circuitry that responds to and processes instructions in program code present in the working memory. Also stored in the non-volatile memory 907 are any components necessary to execute the Script 2, including in this example a built in language library, an interpreter and any third party libraries. When executed, a copy of the Script 2 is 15027066-1 loaded into the working memory, together with any other components necessary to execute the Script 2. The built-in language library, interpreter and third party libraries are the same as described in relation to Figure 1(b). The protection package (protection- layer) comprises a file, __init__.py , which comprises code for addressing vulnerabilities in one or more functions, as discussed below. The Script 2 is shown below: import protection-layer SECRET_API_KEY = '8675309-3141592-6535897' class WebPage(object): def __init__(self, api_key): self.api_key = api_key def render(self, html_template, text): return html_template.format(text) def publish(html_to_publish): “”” Publish the webpage to a server using the API key.""" # Code for publishing omitted # receive html_template from user, e.g. “{text}” html_template = input(“Enter template:”) # receive text from user to insert into template text = input(“Enter text:”) #generate html for the web page from the text provided by the user html = WebPage(SECRET_API_KEY).render(html_template, text) # print the generated html print(html) The Script 2 is the same as Script 1, except for the addition of a single line of code, comprising “import protection-layer”. This line imports all of the functions in the protection-layer package and executes initiation code. The file, __init__.py, in the protection package comprises code defining at least one protected function. In this example, the file comprises code defining the protected function protected_format, but in other examples the file may comprise code that references code elsewhere, such as in another file or an online repository, that defines the protected function. For each of one or more pre-specified functions which have a known potential vulnerability, a corresponding protected function is described by a 15027066-1 definition file in the protection package (the protected functions can be in the same file or in separate respective files, for example). The __init__.py file can comprise initiation code that, when the file is processed by the interpreter, causes the interpreter to generate bytecode, for execution by the (in this case, Python) runtime environment, in which calls in the source code to the one or more pre-specified functions code are replaced in the bytecode by calls to the corresponding protected function(s). In the present example, this is achieved by modifying the symbol table used by the interpreter to generate the bytecode. The symbol table is a data structure (e.g. a dictionary) that is constructed and maintained by the interpreter in order to store information about each of the symbols (or identifiers) used in the program module. For example, the symbol table comprises entries corresponding to each of the functions used in the program module, with each of the entries comprising a respective reference or pointer for calling the corresponding function, e.g. a pointer for calling the corresponding function, or a reference to a data structure or object (e.g. within Python, a PyObject) that comprises the pointer for calling the corresponding function. In the Python programming language, the (global) symbol table can be accessed at runtime using the built-in globals() function. The initiation code modifies the symbol table to replace the reference or pointer of each of the one or more pre-specified functions with the reference or pointer of the corresponding protected function. Following this modification of the symbol table, calls in the source code to the one or more pre-specified functions are resolved, by the interpreter using the symbol table, to the corresponding protected functions. The following code provides one exemplary implementation of this method. The initiation code can, for example, call a function (referred to as “replace_function” below) for each of the pre-specified functions, with an identifier for the pre-specified function and an identifier for the corresponding protected function being provided as inputs (the identifiers may be the unique names of the functions, for example). Modifying the symbol table in this way allows the replacement, in the program module, of the one or more pre-specified functions by the corresponding protected functions without extensive changes needing to be made to the source code of the program module. For example, in the present example, the only change that needs to be made to effect the replacement(s) is the inclusion of the import statement for the protection package. One exemplary implementation of the initiation code is now provided. 15027066-1 def make_hard_copy(org_ref, globals_dict=globals()): copy_name = f"{org_ref.__name__}_copy" globals_dict[copy_name] = copy.deepcopy(org_ref) def adjust_ref_to_copies(func, globals_dict=globals()): """ Make any name that func was called by point to the replacement_func excluding the copy for restore. """ for key in globals_dict: if key == f'{func.__name__}_copy': continue if key == continue if globals_dict[key] == globals_dict[f'{func.__name__}_copy']: globals_dict[key] = globals_dict[f"{func.__name__}"] def adjust_builtins_copy(func, replacement_func): """ if func is a builtin, adjust __builtins__ copy. """ if type(func) is types.BuiltinFunctionType: try: builtin_ref = getattr(__builtins__, func.__name__) builtin_ref = replacement_func except: pass def restore(org_name, globals_dict=globals()): globals_dict[org_name] = \ copy.deepcopy(globals_dict[f"{org_name}_copy"]) def replace_function(original_func, replacement_func, globals_dict=globals()): make_hard_copy(original_func, globals_dict) adjust_builtins_copy(original_func, replacement _func) globals_dict[original.__name__] = override_func adjust_ref_to_copies(original_func, globals_dict) replace_function(format, protected_format) Other implementations of the method are of course possible, including implementations in programming languages other than Python. For example, the modification of the symbol table can be performed during processing of the source code by a compiler to generate machine code in which the calls to the one or more pre- specified functions in the source code are replaced by calls in the machine code to the corresponding protected functions. 15027066-1 When a call to a format function is encountered by the interpreter, the protected_format function is called instead. The protected_format function can be implemented in many different ways. For example, the function can search a string provided as a format specifier for one or more keywords (such as “globals” in the example given above) that are indicative of a malicious input. The function may then, for example, raise an error, or take some other corrective action, when such a keyword is detected. The protection package is thus used to provide a protection layer or protection mechanism. Although in this example, an edit has been made between Script 1 and 2, this edit involves only an initial line of code importing the protection package. This can be added in the same manner to all applications, thus implementing the protection for all applications, without needing to make individual edits to the functions or use of the functions within the application code. Furthermore, no modifications to the existing built- in libraries or third party libraries is made. Rather, the protection is implemented in this example as an additional package. No substantive change is made to the source code of the application being protected (Script 1), nor is there any change to any internal language library or implementation, or any third party library or implementation. The protection layer is therefore non-intrusive regarding the application code and the device 900. In this example, the protection layer can detect whether the use of the function in the code with the particular input string is a security vulnerability, and raises an exception – stopping executing of the code at this point – if so. As well as detecting the vulnerability during execution of the code, it also stops execution of the code if a vulnerability is detected, thus providing protection. Since the protection layer is provided separately to the application code, in this example in a package, it can be used to protect against input string attacks for any application running on the device. In particular, the protection layer can be provided as a module, package or library, or other such unit. It may be quickly implemented on a device 900, regardless of the size of the application code base. Furthermore, it does not require knowledge of the application code itself. In this example, the protection layer implements protection against a format string attack. However, in some examples, the protection layer can offer protection against other vulnerabilities, e.g. more than one vulnerability, by including various protected functions to replace a plurality of pre-specified functions. For example, it may be developed incrementally to include additional vulnerabilities, without losing any of the 15027066-1 previous protections. The protection package may be updated to include additional protected functions as new vulnerable functions are identified, such as by adding additional corresponding definition files. Furthermore, each protected function can be updated to protect against new vulnerabilities found for the function, for example by updating a list of strings that should not appear in the format specifier provided to the format function. The protection mechanism can offer a protection against one or more vulnerabilities in one action, e.g. by importing a protection-layer package. Further details of the device 900 shown in Figure 2(a) will now be described. The device 900 is a computing device such as a server device or an end-user computer. The device 900 receives input data, such as the string arguments for the render function in the Script 2, as detailed above. In some examples, the input data is received from a separate user device (not shown). The input data is then transmitted to the device 900 from the user device through a communication network, for example the Internet. In other examples, a user directly enters the input data, for example through a keyboard connected to or part of the device 900. The input data is received at the input interface 901 of the device 900. Where the input data is received from a separate user device, the input interface 901 is a receiver for receiving data from a communication network, such as the Internet. Alternatively, the input interface 901 may comprise a keyboard or touchscreen for example. The processor 905 accesses the input interface 901. The processor 905 is coupled to the non-volatile storage 907 and also accesses the working memory 911. The processor 905 may comprise logic circuitry that responds to and processes the instructions in program code stored in the working memory 911. In particular, when executed, a program is represented as a software product stored in the working memory 911. Execution of various programs by the processor 905 causes embodiments as described herein to be implemented. In this way, implementations of the embodiments described herein can be realized using one or more modules of computer program instructions. For example, execution of application code (such as Script 2) by the processor 905 causes the protection package to be implemented, and one or more functions of the protection package to be executed on the device 900. The protection mechanism is implemented as a software product, in this case comprising a Python package. The processor 905 is configured to communicate with the non-volatile storage 907. As illustrated, the storage 907 is local memory that is contained in the device 900. 15027066-1 Alternatively however, the storage 907 may be wholly or partly located remotely from the device 900, for example, using cloud based memory that can be accessed remotely via a communication network such as the Internet. Programs are stored in the storage 907 and placed in working memory when executed. In particular, program code corresponding to the protection layer is stored in the storage 907 and placed in working memory when executed. In this example, the program code is stored as a package. The processor 905 also accesses the output interface 903. The output interface 903 provides a response generated by the processor 905 to a communication network such as the Internet. Alternatively, the output interface 903 may comprise a display for example. The input and output interfaces 901, 903 may be a single component or may be divided into a separate input interface 901 and a separate output interface 903. As illustrated, the system 900 comprises a single processor. However, programs may be executed across multiple processing components, which may be located remotely, for example, using cloud based processing. Usual procedures for the loading of software into memory and the storage of data in the storage unit 907 apply. In particular, programs can be embedded in original equipment, or can be provided, as a whole or in part, after manufacture. For instance, programs can be introduced, as a whole, as a computer program product, which may be in the form of a download, or can be introduced via a computer program storage medium, such as an optical disk. Alternatively, modifications to existing software can be made by an update, or plug-in, to provide features of the described embodiments. In particular, the protection layer (in this example the protection package) can be embedded in original equipment, or can be provided, as a whole or in part, after manufacture. For instance, the protection package can be introduced, as a whole, as a computer program product, which may be in the form of a download, or can be introduced via a computer program storage medium, such as an optical disk. Alternatively, modifications to existing software can be made by an update, or plug-in, to provide features of the described embodiments. In particular, the protection mechanism (in this example, the protection package) may be stored locally on the device 900 or it may be provided to the device 900 as a copy of a versioned software solution. While it will be appreciated that the embodiments described herein could be implemented using any computing system, the example device 900 illustrated in Figure 2(a) provides means capable of putting an embodiment, as described herein, into effect. In use, the device 900 receives, by way of input 901, data needed to execute the Script 15027066-1 2. The program Script 2 and protection package, when executed on processor 905, perform a method and provide an output in the manner described herein. The device 900 outputs data by way of the output 903. In Figure 2(b), the device 900 stores a protection package which replaces (“overrides”) calls to certain functions in the built in language library or in a third party library. In this example, a protection layer is implemented as a package. However, in other examples, the protection layer may be in the form of a callable library, may be embedded in the interpreter (as an interpreter plugin or add-on), embedded in the engine (as an engine plugin or add-on), or embedded in a complier (as a complier plugin or add- on). For example, to implement the protection layer, the compiler (interpreter) may be configured to automatically modify the symbol table (as described above in relation to the initiation code) that is generated and used during compilation of the program module. In such cases, no modifications need to be made to the source code of the program module. The protection layer may be provided as a compilation option, an interpreter option, a packaging option, an installation option, or an invocation option. The choice of the form of the protection layer may depend on the language that the protection layer will be used with (e.g. whether the language is interpreted or compiled). The above example describes protection against string attacks for a format function vulnerability. However, protection can also be implemented in a similar manner for other vulnerable functions. For example, the Python function exec() executes specified code. Script 3 is an example of a program which is vulnerable to a format string attack, which includes an exec() function: def addition (a,b): return exec(“%s + %s” % (a, b)) input_1 = result_1 = addition(input_1[‘a’], input_1[‘b’]) In use, a user sends a string to the device 900. The string is received through the input 901. The device 900 inserts the input string into the Script 3 as the input_1, and then executes the Script 3. A legitimate user input is {“a”: “1”, “b”: “2”}. The below shows the Script 3 with this legitimate user input inserted into the Script 3: def addition (a,b): return exec(“%s + %s” % (a, b)) input_1 = {“a”: “1”, “b”: “2”} 15027066-1 result_1 = addition(input_1[‘a’], input_1[‘b’]) A malicious user input is {“a”:”__import__(‘os’).system(‘bash –i >& not_real_address 0>&1’)#”, “b”:”2”} The below shows the Script 3 with this malicious user input: def addition (a,b): return exec(“%s + %s” % (a, b)) input_1 = {“a”:”__import__(‘os’).system(‘bash –i >& not_real_address 0>&1’)#”, “b”:”2”} result_1 = addition(input_1[‘a’], input_1[‘b’]) The malicious user input is used to execute arbitrary code on the device 900. The protection package can include a protected version of the exec() function, a “protected function”. Again, by editing the Script 3 to add an initial line of code, import protection-layer, all of the protected functions in the protection-layer package from the application library can be imported and provide protection against the string attack. The modified script, Script 4 is shown below: import protection-layer def addition (a,b): return exec(“%s + %s” % (a, b)) input_1 = {“a”:”__import__(‘os’).system(‘bash –i >& not_real_address 0>&1’)#”, “b”:”2”} result_1 = addition(input_1[‘a’], input_1[‘b’]) The protection mechanism is implemented on the device that receives the input string and executes the code taking the string as input. Figure 3(a) is a schematic illustration of a system comprising a device 900 according to an embodiment. As shown in Figure 3(a), the input data (in this case, a string) is received at the device 900 from a separate third party device 800. For example, the device 900 may be a server, and the third party device 800 may be an end-user device accessing services provided by the server 900 through a web-browser. In this case, the user inputs the string (for example through a keyboard) at the device 800, using a web-browser running on the third party device 800. The third party device 800 then sends the string to the server device 900. 15027066-1 Figure 3(b) shows a schematic illustration of a system comprising a device 900 according to an embodiment. As shown in Figure 3(b), in this example, a user directly enters the input string, for example through a keyboard connected to or part of the device 900. In both cases, the device 900 executes the program which uses the input data (e.g. string), and also implements the protection. Figure 4(a) shows a schematic illustration of a system comprising a device 900 according to an embodiment. The system comprises the device 900, a separate third party device 800, and a hardware security module (HSM) device 700. The input data (string) is received at the device 900 from the separate third party device 800. The device 900 executes the program which uses the input data (string), and also implements the protection. The program which uses the input data (string) in this example interacts with the HSM 700. For example, the program may be a Python tool allowing a user to remotely connect to the HSM 700 to perform operations. By implementing the protection mechanism on the device 900, mitigation can be provided against a scenario in which a disgruntled employee provides input data designed to retrieve sensitive information from the HSM 700 that they normally do not have access to, or where malicious code has been installed on the device 900. An HSM is a device that securely stores and manages cryptographic keys, and performs a set of cryptographic operations, such as generating a cryptographic key, performing encryption and decryption or performing digital signature and verification operations. Figure 4(c) is a schematic illustration showing an example HSM device 700. The HSM 700 comprises a main processor 202, non-volatile storage 208, random access memory (RAM) 209, crypto co-processor 204, power / reset component 203, input / output component 201, random number generator 205, security processor 206, and one or more sensors 207. A user may store sensitive cryptographic material on the HSM 700. For example, one or more cryptographic application keys are associated with a user of the HSM device. The cryptographic application keys may be stored on the HSM 700. Alternatively, the cryptographic application keys may be encrypted with a master key and stored outside the HSM. The master key, or material that can be used to obtain the master key, is then securely stored inside the HSM. The crypto co-processor 204 performs various cryptographic functions in hardware, for example various standard encryption and decryption algorithms, and 15027066-1 digital signature algorithms. The main processor 202 may offload various operations onto the crypto co-processor 204. The HSM 700 comprises an Input / Output (IO) connector 201. The IO connector 201 is communicatively coupled to the main processor 202 and is configured to act as an interface, through which data is communicated, between the main processor 202 and an external system. The Hardware Security Module (HSM) in this example is configured to be communicatively coupled to the device 900 in an external system through the Input / Output (IO) connector 201. For example, the Hardware Security Module (HSM) can be a PCI express card, which can be directly plugged into the device 900. In this case the Input / Output (IO) connector is a PCIe connector. In use, the Hardware Security Module (HSM) receives user requests through the Input-Output (IO) connector 201. The requests may comprise commands to perform certain cryptographic operations. The Input-Output (IO) connector 201 is communicatively coupled to the main processor 202. The Hardware Security Module (HSM) further comprises non-volatile memory 208 and working memory comprising Random Access Memory (RAM) 209. The non- volatile memory 208 may include any form of non-volatile device memory. In an example, the non-volatile memory 208 includes Flash memory and Electrically Erasable Read Only Memory (EEROM). The RAM 209 may be DDR RAM. The main processor 202 is in wired bi-directional communication with both the non-volatile storage 208 and the Random Access Memory (RAM) 209. The Hardware Security Module (HSM) further comprises a Power / Reset component 203 configured to control the power provided to the main processor 301. The HSM further comprises a random number generator component 205. Computer program code is stored in the non-volatile memory 208. When executed, a program is represented as a software product, or process, in the working memory. The processor 202 comprises logic circuitry that responds to and processes instructions in program code present in the working memory. The below description refers to “firmware”, which is program code comprising a set of computer instructions. The firmware comprises machine code stored in the non-volatile memory 208 on the HSM. Also stored in the non-volatile memory 208 on the HSM are any components necessary to execute the firmware, including runtime system files. When executed, a copy of the firmware machine code is loaded in the working memory. A “firmware process” is the instance of the firmware that is being executed, comprising the machine code in the working memory. The firmware may comprise computer instructions 15027066-1 embodying a set of one or more cryptographic algorithms. For example, the firmware comprises computer instructions embodying one or more of the following cryptographic algorithms: cryptographic key generation; key derivation; encryption; decryption; and digital signature algorithms (for example digital signing or validation of a digital signature). The firmware can be embedded in the non-volatile memory 208 of the Hardware Security Module when the Hardware Security Module (HSM) is manufactured by a trusted party, or can be provided by the trusted party as a whole or in part after manufacture. For instance, the firmware can be introduced by the trusted party as a computer program product, which may be in the form of a download. Alternatively, modifications to existing firmware can be made by the trusted party by an update or plug- in. Execution of the firmware by the processor 202 causes various functionality of a Hardware Security Module (HSM), such as generating cryptographic keys, storing cryptographic keys, or performing cryptographic operations etc., to be implemented. The processor 202 runs an operating system, for example a Linux operating system. The operating system comprises system software that manages the hardware and software resources of the HSM device, and acts as an intermediary between the firmware and the HSM hardware. Figure 5 illustrates a method of improving the security of a program module, according to an embodiment. In step 501 source code for a program module is received by a computer system. For example, the computer system may be one which runs a security enhancement software application which is used to modify the program module. The software application may be supplied to an author of source code, who can use to improve the security of the program module generated from the source code. The computer system may be one configured to perform a compilation process or interpretation process on the software module (e.g. the interpreter / compiler may be provided in an interface of a secure system such as an HSM). The interpreter / compiler may be configured to recognise an initiation command within the program module, and upon doing so to perform steps 502-504 described below. In step 502, the computer system identifies functions called in source code for the program module which are one of a set of pre-specified functions. These may for example be functions for which a security vulnerability operation has been identified, or functions from an undesirable library. 15027066-1 In step 503, the computer system processes the source code to generate, for each call to a function in the source code for the program module, a corresponding program portion of the program module. For at least one of the identified functions, each program portion corresponding to the identified function calls the corresponding replacement function. Each replacement function(s) can, for example, provide the same functionality as the corresponding identified function it replaces, but without one or more vulnerabilities that the identified function has. As another example, the replacement function may provide a wrapper for the identified function, which is configured while the modified program module is run, to provide one or more security operations to recognise situations in which the security of the system may be compromised (as indicated by a determination that one or more vulnerability criteria are met), and to take corrective action when necessary. For example, the replacement function may be operative to determine that input data for one of the identified functions meets a first vulnerability criterion, and / or that output data generated by a function of the program module (e.g. one of the pre-specified functions) meets a second vulnerability criterion, e.g. includes a match to a portion of a database of secure information (e.g. data associated with another user of the secure computer system). In step 504, the (“modified”) program module is output. This may, in some implementations, be in the form of bytecode or machine language, which may be input to an HSM for execution. Figure 6 illustrates a method of protecting against attacks (e.g. string attacks), according to an embodiment. The method is implemented using an isolated protection layer which overrides vulnerability exposed functions of the language or third party libraries. For each such function it performs a series of checks on either the function input and / or the function with the given input before determining if the function with the given input is safe to use. If so, it will return the output of the protected function with the given input. The protection mechanism is implemented as an isolated holistic software tool which may be applied on a code base in order to provide a fast non-intrusive set of (e.g. string) attack protections. A tool built using this method may offer a protection against multiple attack vulnerabilities in one tool implementation. Furthermore, the tool may be updated to newer versions including more vulnerability protections. The tool provides protection against attack vulnerabilities, without making substantive changes to the application code, built in libraries or third party libraries. The tool provides protection 15027066-1 during execution of the code, stopping execution of the code if a vulnerability is detected. The tool offers a solution providing protection, which may be implemented with a large complex code base in a short time without interfering (i.e. changing or examining) the code base being protected or the system it runs on. The tool may be updated to include more vulnerability protections. In an example, the protection mechanism is configured to protect against a Log4j2 vulnerability (CVE-2021-44228). Tools and services may be developed based on the protection layer. In particular, the vulnerability may be fixed only once (e.g. by modifying the symbol table as described above), in the inserted isolated protection layer. For each function being protected, there is no need to know what the function does. The protection layer may be implemented based only on its signature and how a vulnerability might be triggered by input given to it. The protection mechanism may be implemented as a software product. A software product implementing the protection mechanism can be applied on a given code base of another application in order to improve security of the application from one or more string attack vulnerabilities in a fast to protect, easy to apply way. In some implementations (e.g. when the protection layer is implemented as part of the compiler), changes are not made to the code base or any language internal libraries or third party libraries the application uses. The protection mechanism therefore does not interfere with language version or third party version dependencies that the application might have. In particular, the application code is not changed to replace or edit the arguments of the vulnerable functions, or how these functions are used. In one example, the protection layer is implemented as a Python module or pack- age that can be imported into the namespace of client code that is to be protected by the protection layer. The protection layer can comprise code that is executed automatically by the interpreter when the protection layer module or package is imported into the namespace, e.g. using an ‘__init__’ method defined in the protection layer module or package. Thus, in some implementations, the client code can be protected by including an import statement for the protection layer in the client code, i.e. without any other mod- ifications to the client code being needed. In one example, the protection layer module or package comprises code that uses the Python globals() method to obtain a dictionary comprising the functions that have already been imported into the namespace of the client code. The dictionary, which is an example of a symbol table, comprises an entry for each function, each entry com- prising the name of the function and a corresponding reference to an object (in this case 15027066-1 a PyObject) that includes a pointer to the function, i.e. an address in the working memory of the application corresponding to an entry point for calling the function. In programming languages other than Python, e.g. the C programming language, the symbol table can be generated by a compiler and each entry can comprise a pointer to the function (rather than a reference to an object). Returning to the present example, the protection layer code iterates over the functions in the dictionary to determine whether each of the func- tions is a function that is known to have one or more corresponding vulnerabilities, e.g. string attack vulnerabilities. The protection layer code then updates the dictionary ob- tained using the globals() method to modify each entry corresponding to one of the functions determined to have one or more vulnerabilities. In particular, for each of these entries, the reference for calling the vulnerable function is replaced with a reference for calling a corresponding protected function corresponding to the vulnerable function. That is, the protection layer code can obtain, from the dictionary, a reference to an object comprising a pointer to the vulnerable function and replace the reference with a reference to another object that comprises a pointer to the corresponding protected function. Therefore, when the client code attempts to call the function (either directly or by calling another function that calls the function), the call is ultimately directed to the protected function. For example, an object comprising a pointer to the vulnerable function can be obtained using getattr(__builtins__, func.__name__), where __builtins__ pro- vides access a dictionary of the functions in the namespace and func.__name__ iden- tifies the vulnerable function. In some cases, it can be useful to be able to reverse one or more of the modifi- cations to the globals() dictionary, i.e. to revert back to the functions in the namespace before the protection layer module or package was imported or otherwise executed. This can be achieved by storing a (deep) copy of each of the functions having the one or more vulnerabilities and then updating the references for these functions in the dictionary to references for the copy of the function. The protection layer code can use a similar approach to update classes within the namespace, i.e. to replace a class that is associated with one or more vulnerabilities with a “protected” class that has been adapted to handle or fix the one or more vulnera- bilities. In that case, the protection layer code may replace a reference (or pointer) to the vulnerable class with a reference (or pointer) to a protected class instead. In some other cases, it can be preferable to update one or more vulnerable methods of a class without modifying the other methods of the class. This can be implemented by creating a copy of the class (which may be referred to as a “prototype” of the class) and then modifying 15027066-1 or replacing the vulnerable method(s) of the copy of the class. Modifying the vulnerable method can include applying a decorator to wrap the vulnerable method, e.g. using the @functools.wraps(func) method in the Python standard library, where func is the method to be modified or replaced. In this specification, a replacement or “protected” function or method (or class) is a version of the function or method (or class) that may have security advantages over the function, for example one which is adapted to handle or fix one or more vulnerabilities of the function or which is capable of providing an enhanced cryptographic operation. That is, the term “protected” does not refer to an inheritance protection level in object oriented programming languages, i.e. to a protected function or method of a class that is accessible by a subclass of the class. 15027066-1

Claims

Claims 1. A computer-implemented method of modifying a program module including a plurality of program portions which each call a corresponding function, the method comprising: identifying functions called in source code for the program module which are one of a set of pre-specified functions, each pre-specified function being associated with a corresponding replacement function for calling in place of the pre-specified function; processing the source code to generate, for each call to a function in the source code for the program module, a corresponding program portion of the program module, wherein for at least one of the identified functions, each program portion corresponding to the identified function calls the corresponding replacement function; and outputting the program module, wherein processing the source code to generate the program portions of the program module comprises: processing the source code for the program module to generate a symbol table that comprises a respective entry for each of the functions called in the source code, each entry comprising a corresponding reference or pointer for calling the function; replacing the reference or pointer in the entry for the at least one of the identified functions with a reference or pointer for calling the corresponding replacement function; and using the symbol table to generate the program module.

2. The method of claim 1, further comprising storing a copy of the at least one of the identified functions, and wherein the corresponding replacement function is configured to call the copy of the at least one of the identified functions.

3. The method of claim 1 or claim 2, wherein the replacement function is configured to perform a run time monitoring of the corresponding identified function to determine whether a vulnerability criterion is met, and, if the criterion is met, to perform a corrective action. 15027066-14. The method of any preceding claim, further comprising determining whether the identified function meets a vulnerability criterion, and wherein each program portion that calls the replacement function corresponding to the identified function is generated in response to the determination being positive.

5. The method of claim 3 or claim 4, wherein the vulnerability criterion is one or more of: a vulnerability criterion dependent upon the arguments of the identified function; a vulnerability criterion dependent upon outputs of the identified function; a vulnerability criterion dependent on a time taken by the identified function to operate; and a vulnerability criterion dependent on a specified resource being accessed or modified by the identified function.

6. The method of any preceding claim in which the pre-specified functions are functions which have been identified as having a security vulnerability.

7. The method of any preceding claim, in which the identified function and the corresponding replacement function each comprise an instruction to perform a function on one or more arguments recursively for a plurality iterations, the cardinality of the plurality of iterations for the replacement function being different to a cardinality of iterations for the identified function.

8. The method of any preceding claim, which comprises determining whether the arguments of each identified function references one of a set of resources, and upon determining that the arguments of one of the identified functions references one of the set of resources, preventing the one of the set of resources from being accessed. 15027066-19. The method of any preceding claim, which is performed by an interface of a hardware security module, HSM, device, the program module being a module of a software application which is provided to be run on the hardware security module device.

10. The method of any preceding claim, which is performed in response to the source code for the program module being loaded into an interpreter or compiler, the interpreter or compiler being configured to recognise an initiation command within the source code of the program module, and to perform the method upon recognizing the initiation command.

11. The method of any preceding claim, wherein processing the source code to generate the program portions comprises generating corresponding bytecode or machine code for execution by a runtime environment or processor.

12. The method of any preceding claim comprising providing the program module to a runtime environment or processor to be run.

13. The method of any preceding claim, wherein: the replacement function is configured to, as the program module is run, determine whether realisations of arguments of the corresponding identified functions satisfies a vulnerability criterion, and if the vulnerability criterion is met, taking a corrective action; and / or the replacement function is configured to compare outputs generated by the identified function when it is called with a database of secure information to identify matches between the outputs of the program module, and upon identifying a match taking a corrective action; and / or the method further comprises monitoring a time taken by the replacement function to run, and, upon the run time exceeding a threshold, taking a corrective action and optionally, wherein the corrective action includes any one of more of: suspending or terminating the operation of the modified program module; or 15027066-1issuing an alert to a system administrator of a computer system running the modified program module.

14. A computer program product comprising program instructions which, when implemented by one or more processors, cause the one or more processors to perform a method according to any preceding claim.

15. A computer system comprising one or more processors, and a data storage device storing program instructions which, when implanted by the one or more processors cause the one or more processors to perform a method according to any of claims 1 to 13. 15027066-1

Images