Monitoring proper operation of programs in a secure environment

Abstract

In a computer system in which multiple programs are executed concurrently in respective isolated execution environments, the corresponding resources of the computer system consumed by each of the isolated execution environments is monitored, and any of the isolated execution environments for which the consumed corresponding resources meet an excessive use criterion is identified. If any such isolated execution environment is identified, a corrective action is performed. This may be to issue an alert to an administrator of the system, who may in response modify (e.g. suspend or terminate) the operation of the identified isolated execution environment(s). Alternatively or additionally, the operation of the identified isolated execution environment(s) may be automatically modified (i.e. without the intervention of the administrator), e.g. leaving other of the isolated execution environments still operating.

Description

[0001] Monitoring proper operation of programs in a secure environment

[0002] Field

[0003] The present invention relates to a method and device for secure execution of computer programs. The device may be a Hardware Security Module for example.

[0004] Background

[0005] A Hardware Security Module (HSM) is a device that securely stores and manages cryptographic keys, and performs a set of cryptographic algorithms. In some applications, it can be desirable to allow a user of the HSM to perform user-defined algorithms. In this way, a user is able to run a custom program that uses their cryptographic keys on the HSM. However, enabling a user to run a custom program on the HSM could compromise the security or operation of the HSM, and the information stored on the HSM. There is a continuing need to improve the security of such devices.

[0006] Summary of the Invention

[0007] In general terms, the present disclosure proposes that in a computer system in which multiple programs are executed concurrently in respective isolated execution environments, the amount of one or more corresponding resources of the computer system consumed by each of the isolated execution environments is monitored, and any of the isolated execution environments for which the consumed corresponding resources meet an excessive use criterion for the isolated execution environment is identified.

[0008] If any such isolated execution environment(s) are identified, a corrective action is performed without interrupting the operation of all the isolated execution environments.

[0009] This corrective action may be to issue an alert to an administrator of the system, who may in response issue a command to the HSM to modify (e.g. interrupt) the operation of the identified isolated execution environment(s). The HSM may be configured to receive and implement a command from the administrator to modify the operation of the isolated execution environment specified in the command without interrupting the operation of all the isolated execution environments.

[0010] Alternatively or additionally, the operation of the identified isolated execution environment(s) may be modified (e.g. interrupted) automatically, i.e. without the intervention of a human administrator, while leaving other of the isolated execution environments still operating.

[0011] In either case, the other isolated execution environments may be substantially unaffected by the modification to the isolated execution environment, other than by being no longer starved of resources or being in any other way adversely affected by the excessive use by the identified isolated execution environment.

[0012] This has an advantage that any rogue or faulty isolated execution environments meeting the corresponding excessive user criterion may be identified and corrective action can be taken. In this way Denial of Service (DOS) attacks and side channel attacks may be prevented, or at least reduced, as well as system degradation due to faulty containers. Thus, any temperature limitation, or power limitations, or other finite resource requirements (e.g. memory or access to processors) of the computer system as a whole may be met, and / or it can be ensured that properly operating isolated execution environments are not undesirably starved of appropriate resources, while reducing unnecessary disruption to the properly operating isolated execution environments. This is particularly advantageous in situations in which there are multiple tenants, which may include both bona fide users and malicious users (rogue actors) causing corresponding programs to be run concurrently on the HSM.

[0013] Brief Description of the Figures

[0014] Devices and methods in accordance with non-limiting embodiments will now be described with reference to the accompanying figures in which:

[0015] Fig. 1 is a schematic illustration of an exemplary Hardware Security Module (HSM) device;

[0016] Fig. 2 is an illustration of the logical structure of the HSM;

[0017] Fig. 3A shows a container-based virtualization stack of the HSM device according to a comparative example;

[0018] Fig. 3B shows a process performed by the HSM device according to a comparative example to initiate an isolated execution environment; Fig. 4 illustrates schematically the operation of a system which is an example of the present disclosure;

[0019] Fig. 5 is a method carried out in the a system which is an example of the present disclosure;

[0020] Fig. 6 shows an assignment of a threshold value of an corresponding excessive use criterion to differing containers in a possible implementation of the system of Fig. 4;

[0021] Fig. 7 shows the logical structure of a system to implement the operation of Fig. 4;

[0022] Fig. 8 is an illustration of the logical structure of the HSM in another system which is an example of the present disclosure;

[0023] Fig. 9 shows a container-based virtualization stack in the example system of Fig. 8;

[0024] Fig. 10 illustrates a method performed by the example system of Fig. 8;

[0025] Fig. 11A illustrates schematically the operation of a first possible isolated environment initialization process performed by the example system of Fig. 8;

[0026] Fig. 11 B illustrates schematically the operation of a second possible isolated environment initialization process performed by the example system of Fig. 8;

[0027] Fig. 12 illustrates schematically associations between multiple users and multiple certificates in the example system of Fig. 8;

[0028] Fig. 13 illustrates two certificates associated with one of the users in the example system of Fig. 8;

[0029] Fig 14A illustrates a method of constructing a certificate for use in the example system of Fig. 8;

[0030] Fig. 14B illustrates a method of associating a user with a certificate in the example system of Fig. 8; Fig. 15 illustrates a method of receiving a request to execute a program and implementing the request in the example system of Fig. 8;

[0031] Fig. 16 illustrates a first way of implementing a selection step of the method of Fig. 15;

[0032] Fig. 17 illustrates a second way of implementing a selection step of the method of Fig. 15; and

[0033] Fig. 18 illustrates a further method performed by the example system of Fig. 8.

[0034] The same reference numerals are used to designate corresponding elements in the illustrations of the comparative example and the example of the disclosure.

[0035] Detailed Description

[0036] A Hardware Security Module (HSM) is a device that securely stores and manages cryptographic keys, and performs a set of cryptographic algorithms. In some applications, it can be desirable to allow a user of the HSM to perform user-defined algorithms. In this way, a user is able to run a custom program that uses their cryptographic keys on the HSM. However, enabling a user to run a custom program on the HSM could compromise the security or operation of the HSM, and the information stored on the HSM.

[0037] Mitigating against such security issues can be important in a multi-tenant environment for example, where the HSM stores the cryptographic keys for multiple different users, who may not necessarily consent to their keys being shared with the other users.

[0038] In order to mitigate against such security issues, the user-defined program is executed in an isolated environment. For example, a container is used to execute and isolate a user-defined program.

[0039] The containers may be initialised with a standard configuration setting. For example, the configuration settings may be based on a whitelist provided by a trusted party, for example the manufacturer of the HSM. The whitelist may specify one or more of: the system calls that are considered safe for use by a user, the available devices that are considered safe for use by a user, the operating system capabilities that are considered “safe”. However, using a standard whitelist of configuration settings can result in a user- defined program being executed in a container that allows more functionality than the program needs in order to run. This can compromise the security or operation of the HSM, and the information stored on the HSM.

[0040] Fig. 1 shows a schematic illustration of a Hardware Security Module (HSM) device 11 in accordance with an example. The HSM 11 comprises a processing unit 103, a nonvolatile memory 105 and working memory comprising Random Access Memory (RAM) 109.

[0041] In this example, the processing unit 103 is a Central Processing Unit (CPU) 103, however in other examples, the processing unit may comprise a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC) for example, or a combination of multiple types of processing components.

[0042] The non-volatile memory 105 may include any form of non-volatile device memory such as flash, optical disks or magnetic hard drives for example. The non-volatile memory 105 may be physically secure and may be resistant to tamper by a third party, for example by the inclusion of physical security such as a membrane that covers the entire device, that cannot be removed without destroying the underlying physical hardware. The CPU 103 is in wired bi-directional communication with non-volatile memory 105 and the RAM 109.

[0043] Computer program code is stored in the non-volatile memory 105. When executed, a program is represented as a software product, or process, in the working memory. The processor 103 comprises logic circuitry that responds to and processes instructions in program code located in the working memory. The working memory comprises the RAM 109.

[0044] The below description refers to “HSM firmware” (or “main application”), which is a program comprising a set of computer instructions. The HSM firmware comprises machine code stored in the non-volatile memory 105 on the HSM 11. Also stored in the non-volatile memory 105 on the HSM 11 are any components necessary to execute the HSM firmware, including runtime system files. When executed, a copy of the main application machine code is loaded in the working memory. A "main application process” is the instance of the HSM firmware that is being executed, comprising the machine code in the working memory. The HSM firmware comprises computer instructions embodying a set of one or more cryptographic algorithms. For example, the HSM firmware comprises computer instructions embodying one or more of the following cryptographic algorithms: cryptographic key generation; key derivation; encryption; decryption; and digital signature algorithms. The HSM firmware can be embedded in the non-volatile memory 105 of the Hardware Security Module 11 when the HSM 11 is manufactured by a trusted party, or can be provided by the trusted party as a whole or in part after manufacture. For instance, the HSM firmware can be introduced by the trusted party as a computer program product, which may be in the form of a download. Alternatively, modifications to existing HSM firmware can be made by the trusted party by an update or plug-in.

[0045] The HSM 11 runs an operating system, for example a Linux operating system. The operating system comprises system software that manages the hardware and software resources of the HSM 11 , and acts as an intermediary between the HSM firmware and the HSM hardware.

[0046] In use, the HSM device 11 may be located within a larger system. In this case, the HSM 11 is communicatively coupled to a computer or a server device in the larger system (for example) through interface 107, which comprises a communication link to the computer or server device. For example, the HSM 11 can be implemented in the form of a PCI express card directly plugged into the computer or server device, or the HSM device 11 can be communicatively connected to the computer or server device by a USB connection. In use, the HSM device 11 receives user requests through interface 107.

[0047] In one example, the HSM 11 is located in a user system. In this case, the user system has a dedicated local HSM 11 device. The HSM 11 is directly coupled to a client (user) computing device 117 (controlled by the user) through interface 107, for example the HSM device 11 can be a PCI express card directly plugged into the computing device 117 or can be communicatively connected to the client computing device 117 by a USB connection. The client computing device 117 may be a server or an end-user computing device for example.

[0048] In a different example, the HSM 11 is located in a host system (not shown) which is separate to the client computing device 117. The host system may be a service provider system for example. In this case, the client computer 117 communicates with a computer or server device in the host system. The HSM device 11 is directly coupled to the computer or server device in the host system through interface 107. For example, the HSM device 11 can be a PCI express card directly plugged into the computer or server device in the host system, or the HSM device 11 can be communicatively connected to the computer or server device by a USB connection. The client computing device 117 is thus communicatively coupled to the HSM device 11 through a computer or server device present in the host system (which is not shown in Fig. 1). Communication between the user computing device 117 and the host computer or server device (not shown in Fig. 1) may be performed via an Internet connection for example.

[0049] The non-volatile memory 105 stores one or more cryptographic keys associated with a user. The one or more cryptographic application keys are associated with the user for use with the cryptographic algorithms embodied in the HSM firmware. The application key(s) may be securely stored outside of the HSM device 11 using a master key. For example, the application key(s) may be encrypted using Advanced Encryption Standard (AES) encryption (for example) using the master key stored in the HSM device. The encrypted application key(s) can then be transmitted to a separate external device for storage. The master key however is stored on the HSM device 11 .

[0050] The application keys associated with the user are used to perform one or more cryptographic algorithms embodied in the HSM firmware. When a user request to perform an algorithm is received at the HSM 11 , the relevant application key or keys are retrieved by the HSM 11 , decrypted using the master key, and loaded into the RAM 109 space of the HSM firmware process. They may then be used by the HSM 11 to perform one or more cryptographic algorithms, including performing various encryption and decryption algorithms, and performing digital signature algorithms.

[0051] The HSM device 11 further comprises a Board Support Processor 113. Board Support Processor 113 is configured to communicate with a plurality of on-board sensors, monitoring the operation of the main CPU and the other hardware components of the Hardware Security Module 11 .

[0052] The sensors may include, but are not limited to, temperature sensor(s) 114 and / or power sensor(s) 115 such as voltage sensors and / or current sensors. Each of the temperature sensors 114 and / or power sensors 115 may relate to a specific portion of the HSM 11. For example, the output of a certain one of the temperature sensor(s) 114 may indicate the temperature of one or more corresponding devices of the HSM 11 (e.g. the CPU 103) or a certain location on the circuit board of the HSM. Similarly, the output of a certain one of the power sensor(s) 115 may characterize the power consumed by a corresponding one or more of the other devices of the HSM 11 .

[0053] The HSM device 11 further comprises a power reset device 116, configured for control by any one or more of the CPU 103, the crypto co-processor 111 or by a device external to the HSM 11 (e.g. using commands transmitted to the power reset device 116 via the interface 117).

[0054] The HSM further comprises a random number generator component 104, such as a quantum entropy source.

[0055] In some examples, the HSM device 11 may further comprise a crypto co-processor 111. The crypto-coprocessor 111 may perform various standard cryptographic functions in hardware, for example various encryption and decryption algorithms, and digital signature algorithms. In other examples, this functionality is integrated in the main processing unit 103. For example, the main processing unit 103 may comprise a system on a chip FPGA (SoC FPGA), where the various standard cryptographic functions are implemented in hardware in the programmable logic.

[0056] Various other components may be included which are not shown in Fig. 1.

[0057] Although a HSM device is described here, the invention is not limited to such, and the below described methods may be implemented on various devices used for performing cryptographic algorithms and / or storing cryptographic keys. For example, a software emulator running on a general purpose computing device or server may enable the computing device or server to behave like a HSM device. In this case, a computer program running in the computer device or server emulates a HSM device, and the below described methods are implemented on the computer device or server in this environment.

[0058] As discussed above, the HSM firmware comprises computer instructions embodying a set of one or more cryptographic algorithms. However, a user may wish to perform an algorithm for which computer program code is not provided in the HSM firmware on the HSM device 11. In light of this, there is provided functionality to receive and execute a user-defined program at the HSM 11 , as will be described in more detail below.

[0059] The user-defined program may be specified in a form that can be executed by the CPU 103 (e.g. in machine code). In other examples, the user-defined program is provided to the HSM 11 in an intermediate form, and is then converted by the HSM 11 into a form that can be executed by the CPU 103. Enabling user-defined code to be executed on the HSM provides flexibility, and allows a user to execute custom algorithms on the HSMs that make use of their cryptographic keys.

[0060] However, allowing a user to unconditionally execute arbitrary user-defined code on a Hardware Security Module (HSM) 11 may compromise the operation and security of the HSM 11 . The user code is therefore executed in an isolated environment on the HSM device 11 . For example, the HSM 11 uses containers to execute user code. A container is an example of an isolated environment. Using the container to execute the code isolates the user-defined code from the rest of the HSM runtime environment. A container is an executable unit of software in which application code is packed along with its libraries and dependencies. Containers take advantage of a form of Operating System (OS) virtualization in which features of the OS are leveraged to isolate processes. The containers may be referred to as OS-level containers because they are managed by the OS.

[0061] Isolating the user-defined program from the rest of the run-time environment creates a consistent environment for executing user-defined programs and limits exposure to rest of the HSM runtime environment. Executing the user-defined code in a container thus provides a sandbox in which the execution environment can be controlled and isolated from the HSM firmware. This mitigates against a user accidentally or deliberately introducing a security vulnerability to the HSM.

[0062] As discussed above, in an example the HSM 11 runs the Linux operating system. The Linux operating system may support the use of Linux containers (e.g. Linux LXC containers, and / or Linux LXD containers). Where Linux containers are used, Linux container technology provides the ability to control the operation of the container, e.g. by limiting the operations that can be performed by the container. The configuration of a container can be controlled to limit the operations that can be performed by code being executed in the container. The configuration associated with each Linux container is defined in a configuration file. For example, the container can be configured in such a way that it limits the system calls to the Operating System (OS) that can be made by the code executed in the container. Linux has over 300 system calls. Allowing a user defined program to access all 300 system calls could give rise to a security vulnerability. To this end, the system calls that can be made by programs executing in a container may be limited to a “whitelist” of system calls - this may be around 100 system calls for example. These are the system calls that are considered to be “safe” by a trusted party, for example a system administrator. Restricting the number of system calls reduces the attack surface, i.e. the potential ways in which the security of the HSM can be compromised. The available system calls can be limited using a seccomp profile, which specifies the allowed system calls. The seccomp profile is used to configure a seccomp filter when the container executing the user-defined program is initialised. The container can additionally or alternatively be configured in such a way that it limits the devices (e.g. storage devices) that can be accessed by the code executed in the container for example.

[0063] By controlling the configuration of the containers used to execute user code, even if the user-defined code contains malicious elements, the effect it could have on the security of the HSM (in particular the security of the information, such as cryptographic keys, stored on the HSM) is limited. For example, if the user-defined code has been compromised by a malicious third party, without knowledge of the user, the security issues caused by running the code on the HSM are limited by isolating the user code in the isolated environment (the container in this example).

[0064] This can be particularly important in a multi-tenant environment where the HSM 11 has access to (e.g. stores) cryptographic keys associated with other users. In this case, without sufficient controls in place, a user-defined program from a first user of the HSM 11 could be used to compromise the cryptographic keys of another user.

[0065] Fig. 2 shows a logical diagram of a Hardware Security Module 11 according to a comparative example. In particular, Fig. 2 shows an illustration of various software processes being executed by the HSM 11 . The HSM 11 may be a HSM 11 such as described in relation to Fig. 1 for example.

[0066] As discussed above, a container 203 is a virtual runtime environment that runs on top of an operating system (OS) kernel. The container 203 isolates one or more running processes running in the container 203 from the rest of the system. The container 203 is an example of an isolated environment.

[0067] In this example, a file 201 corresponding to the container 203 is provided by the user in a request transmitted to the HSM 11 . This file may also be referred to as the container image 201 or the container repository. Once received, the container image 201 is stored in the non-volatile memory 105 of the HSM 11. When run, the container image 201 becomes the container 203. The container image 201 comprises program code corresponding to the programs to be run in the container. In one example, once created, the container image 201 is immutable (i.e. such that it cannot be changed). The container image 201 further comprises all libraries, supporting files and programs, and any language interpreter required to run these programs on the HSM 11 . This means that when the container 203 is running, the programs may run without requiring access to files outside the container 203.

[0068] The launcher service 210 comprises a set of program instructions that, when executed, store the received container image 201 in the non-volatile memory 105 in response to receiving a command to start the container 203, and initiate and run the container 203 by sending an instruction to the container engine 202 to initiate a container 203 according to the container image 201 and a configuration file 207. The container configuration file 207 may be a Ixc.conf file for example.

[0069] The client device 117 communicates with the launcher service 210 through a first secure channel. When a request comprising a container image 201 is received from the client device 117 through the first secure channel, the launcher service 210 stores the received container image 201 in non-volatile memory 105, then transmits an instruction to a container engine 202 to extract the container image 201 in the non-volatile memory 105, initialize a container according to the container image 201 and a container configuration specified in a configuration file 207.

[0070] The configuration file 207 may be a default configuration file stored on the HSM 11 or may be generated by the launcher service 210 in response to the request in the manner described in more detail below. The launcher service 210 takes the configuration file 207 and a request to initiate a container from the user, and instructs the container engine 202 to start a container. The configuration file 207 and container image 201 are provided to the container engine 202 by the launcher service 210. A container for executing the user-defined program is thus initialised based on a container image 201 and a configuration file 207. The container configuration file 207 defines, amongst other things, the capabilities of the container, the devices that are accessible to the container, and a seccomp filter that controls the system calls that can be made by programs executing in the container.

[0071] The container engine 202 is configured to initiate the container 203 in response to a request from the launcher service 210. The container engine 202 processes the configuration settings provided in the configuration file 207 and sends this information to the OS kernel to set up the container. The container engine 202 makes one or more API calls to the Operating System (OS) kernel. The OS kernel implements the isolation of the container 203, and enforces the part of the non-volatile memory 105 and network connections available to the container 203. The OS kernel, together with the memory management unit within the CPU 103, allocates part of the RAM 109 to the container, and prevents writing to other parts of the RAM 109. In this way, the container isolation is implemented by the OS kernel, using hardware components such as the memory management unit. Consequently a container is an OS-level virtualisation tool. Once running, a container 203 is a set of one or more processes that are isolated from the rest of the system. Example implementations of the container engine 202 include Linux LXC and LXD.

[0072] The configuration settings specified in the configuration file 207 are implemented by and enforced by the OS kernel. The OS kernel is configured to process requests from the container 203 (e.g. system calls) and action these requests depending on the configuration settings of the container (e.g. whether a specific system call is enabled for the container). The configuration settings of the container are maintained whilst the container is running, and can only be modified by an authorised user, as described below.

[0073] Once running, communication between the client device 117 and the container 203 may be performed via a firewall, and a virtual private network or third secure channel 206 configured by the launcher service 210. The third secure channel may be configured by the user. The HSM 11 also executes a main application process (HSM firmware) 204. The HSM firmware has been described previously, and comprises computer program code for managing, generating and manipulating cryptographic keys. For example, the main application process includes one or more of: predefined algorithms for generating cryptographic keys, code for loading a cryptographic key onto the HSM 11 , code for retrieving a cryptographic key stored on the HSM 11 etc. The firmware 204 may include a root certification authority (CA) certificate, which is accessed when the firmware is used. The HSM firmware 204 is configured to access security world data 205 stored in the non-volatile memory 105 of the HSM 11. Security world data 205 includes cryptographic keys associated with a user. Communication between the client device 117 and the HSM firmware 204 is performed via a second secure channel. In this example, the second secure channel is implemented using the secure shell (SSH) protocol.

[0074] The HSM firmware 204 may communicate with the container 203, to provide data (e.g. security world data 205) for use by the user processes executed within the container 203. For example, the HSM firmware 204 communicates with the container 203 via an internal network port of the container 203. In an example, the container 203 is used to perform a user-defined algorithm that uses a cryptographic key associated with the user.

[0075] The HSM main application code defining the HSM firmware 204 is provided by a trusted party. For example, the main application code can be embedded in the non-volatile memory when the HSM 11 is manufactured, or can be provided by the trusted party by an update or plug-in. However, a user may wish to perform a function that is not provided in the HSM firmware. In the example of Fig. 2, the container 203 is provided for executing a user-defined program. This allows a user access to increased functionality, whilst limiting exposure of the HSM 11. In particular, since the user-defined program is executed in the container 203, the runtime environment of the user-defined program is isolated from the HSM firmware 204 which has access to the security world data 205, thereby improving security.

[0076] Fig. 3A shows a container-based virtualisation stack according to an example. In particular, Fig. 3A shows an Operating System kernel 302 running on the HSM hardware 301 (e.g. as depicted in Fig. 1). The OS kernel 302 runs the container engine 202, the launcher service 210, and the main application 204 (HSM firmware) as separate processes. In the examples described below, the Operating System (OS) is a Linux Operating system.

[0077] The container engine 202 runs the container 203. The container 203 comprises the user process(es) and the supporting files for these processes. In the example of Fig. 3A, the runtime environment of the container 203 is isolated from the runtime environment of the HSM firmware 204. Various known container engines may be used. Examples of container engines include Docker, Linux LXC and Linux LXD.

[0078] As has been described previously, the container 203 is associated with a configuration file 207, which is stored separately to the container image 201 in this example. The configuration file 207 defines one or more of: 1) how the network is virtualized in the container (e.g. which of the host's network interfaces are accessible to the runtime environment of the container); 2) a system call filter (e.g. which controls the system calls that can be made by a program executing in the container); 3) the Linux Capabilities available to the program executing in the container (e.g. controlling whether access is provided to specific kernel resources that were previously unavailable to unprivileged processes); and 4) the hardware devices mounted in the OS file system that can be accessed (e.g. for read or write) by the programs executing in the container 203. The system may support multiple substantially-identical containers concurrently, each processing different requests associated with different container images 201 .

[0079] A simplified schematic form of Fig. 2 is shown in Fig. 3B. Upon a user transmitting to the HSM a request to execute a program, the launcher service 210 initiates the container 203 based on the configuration file 207. The container 203 executes the program while communicating with the container engine 202 and the HSM firmware 204.

[0080] One approach is to initialize all containers for running user-defined code according to a default whitelist of system calls, devices, and capabilities, which is the same for all initialised containers. The whitelist corresponds to a default configuration file stored on the HSM. For example, the whitelist contains the system calls, devices and capabilities that are deemed to be “safe” by a system administrator.

[0081] However, this whitelist may be too permissive for a user’s specific use case. For example, although the number of system calls that can be made may be reduced to around 100 in the default whitelist, even this reduced set of 100 system calls may be too permissive for some use cases.

[0082] Although only a single container is shown in Fig. 3A, it is known for an HSM 11 to run multiple containers concurrently on the HSM hardware 311 . For example, before a given container has finished executing the programs within it, one or more other requests may be received to execute a program, and in response the HSM 11 may initiate a new container in respect of the programs in the new request(s) before the previously8 initiated containers have finished executing their corresponding program(s). The new container is launched in the manner described above, and runs on the container engine in the same way as the container described, in parallel with the existing containers.

[0083] More generally, there may be any number of concurrently running containers.

[0084] The resources of the HSM are limited. For example, to protect the hardware of the HSM device, the power consumption of the HSM and the temperature of the HSM (due to a usage of resources which generates heat) may be subject to constraints, Furthermore, the memory and CPU, which are shared by all the containers, are subject to limits. Future HSMs may be subject to tighter constraints on system resources, e.g. mobile, battery- powered HSMs.

[0085] Unfortunately, situations sometimes arise in which one for the programs running on the HSM consumes excessive resources. This can occur for example in situations in which a container executing a program malfunctions. Furthermore, a malicious actor may desire to compromise the security and / or performance of the HSM.

[0086] For example, if a certain one of the containers uses almost all of the memory or CPU, all the other containers would be unable to access sufficient resources, e.g. pre-determined amounts of the resources which were used when their configuration files were designed. That is, these other containers would be “starved” of resources. This may be the intention a malicious actor associated with the certain one of the containers: to starve the other containers (e.g. associated with different tenants / users) of resources in a denial of service attack.

[0087] One known approach to this is to monitor the requests, traffic logs, voltage and temperatures of the HSM 11 , to determine whether the HSM is being used in a way which is outside its operating parameters (e.g. power usage / temperature) or whether any of the resources is being used beyond a predetermined limit, such that any container is being starved of resources. If such a situation is detected, the operation of the HSM 11 is interrupted. That is, all the containers cease to operate.

[0088] This is undesirable for the tenants / users associated with the containers which operated properly. For future HSMs having tighter operating requirements, shutting down the HSM 11 may occur more frequently, and so will be a more significant problem.

[0089] A first aspect of the disclosure is computer-implemented method comprising:

[0090] (i) receiving, by a computer system, from one or more users one more requests to execute one or more programs (e.g. from each of multiple users, a respective request to execute a respective program);

[0091] (ii) initializing, by the computer system, a plurality of isolated execution environments (containers); (iii) executing, by the computer system, the one or more programs concurrently in the isolated execution environments;

[0092] (iv) determining for each of the isolated execution environments one or more corresponding resource measures indicating resources of the computer system employed by the isolated execution environment;

[0093] (v) identifying any of the isolated execution environments for which the one or more corresponding determined resource measures meet a corresponding excessive use criterion; and

[0094] (vi) upon identifying any of the isolated execution environments for which for which the one or more corresponding determined resource measures meet the corresponding excessive use criterion, performing a corrective action in respect of the identified isolated execution environment without interrupting the operation of all the isolated execution environments.

[0095] At least step (v) may be performed by a unit of the computer system referred to as a “shared resource monitor”. The shared resource monitor may also perform step (iv), i.e. determine at least some of the resource measures. The shared resource monitor may perform, or at least trigger, step (vi) when it identifies any of the isolated execution environments in step (v).

[0096] Conversely, if there is no isolated execution environment (container) for which the corresponding one or more determined resource measures meet the corresponding excessive use criterion (i.e. upon a negative determination that the corresponding resource measure(s) for any of the isolated execution environments meet the corresponding excessive use criterion), no corrective action may be triggered. For example, no modification to the operation of any of the isolated execution environments may be made.

[0097] The corrective action may lead to the operation of the identified isolated execution environment being modified. For example, the operation of the identified isolated execution environment may be interrupted, i.e. suspended or terminated. Alternatively, the operation of the identified isolated execution environment may be modified without interrupting the operation, e.g. by reducing (“throttling”) the resources available to the isolated execution environment. The corrective action may be performed automatically, e.g. it may include, without human involvement, modifying the identified isolated execution environment, without interrupting (suspending or terminating) the operation of all the isolated execution environments.

[0098] Alternatively, the corrective action may comprise providing an indication of the identified isolated execution environment, and optionally other data such as the determined resource measure(s) to a human administrator (e.g. via a user interface of a system in communication with the HSM). In this case, the human administrator may be empowered (e.g. by operation of a user interface) to transmit a command to the HSM instructing the HSM to modify the operation of the identified execution environment, without interrupting (suspending or terminating) the operation of all the isolated execution environments. For example, the administrator may be given a choice of possible modification options, of which at least one does not interrupt the operation of all the isolated execution environments.

[0099] The computer system may be configurable, e.g. by a system administrator, to determine which corrective action is taken upon it being determined that an excessive use criterion is met, e.g. such that a user (system administrator) can select the degree of authority of the shared resource monitor. The corrective action in response to determining that an excessive use criterion is met may be configured as part of the HSM installation.

[0100] In any case, the corrective action may allow continued operation of the other isolated execution environments (the ones for which the corresponding resource measure(s) did not meet the excessive use criterion), thereby reducing unnecessary disruption to those other isolated execution environments. Thus, the operation of the isolated execution environment(s) other than the identified execution environment, may not be “directly” modified, i.e. not modified apart from no longer being subject to the same extent to any ill-effects caused by the identified isolated execution environment.

[0101] Once the corrective action is taken, the method may terminate. Alternatively, steps (iv)- (vi) may be performed repeatedly, e.g. at intervals (e.g. at regular intervals, and / or until a performance of steps (iv)-(vi) is triggered by another event, such as a user command, or the initiation of a new isolated execution environment). The excessive use criterion corresponding to a given isolated execution environment (container) may include (i.e. be defined based on) a set of multiple corresponding tests relating to different respective ones of the determined resource measures for the isolated execution environment. Each test may be based on a corresponding threshold which is a numerical value relating to the resource measure. Determining whether the excessive use criterion is met comprises determining whether the corresponding tests are met, and determining whether the excessive use criterion is met based on the results. Examples of these tests are given below. For simplicity here, it will generally be assumed that the excessive use criterion is met if it is determined that any one of the tests is met (e.g. if a temperature test is met indicating that a temperature of portion of the computer system associated with the container is above a temperature threshold). However, in principle, the excessive use criterion may be determined to be met if any if any one of a plurality of subsets of the set of tests is met. For example, the excessive usage criterion might be determined to be met only if: (i) at least one test of memory usage is met or (ii) if both a temperature test and a power usage test are met.

[0102] As suggested above, the excessive use criterion corresponding to a given container may include at least one temperature test defined by a temperature threshold. For example, the excessive use criterion may comprise at least one temperature test of whether a measured temperature of a location of the computer system associated with the isolated execution environment (e.g. a temperature of a processor employed by the isolated execution environment) is above a temperature threshold associated with the temperature test.

[0103] For example, temperature measurements may be performed at multiple locations in the computer system, and for each location a corresponding temperature test may be performed of whether the corresponding measurement is above a corresponding temperature threshold (which need not be the same in all the locations). The excessive use criterion for any given isolated execution environment may then be based on whether the temperature tests for location(s) associated with the isolated execution environment are met.

[0104] Optionally, the temperature measurements may be recorded in a temperature log, and a given temperature test may be based one temperature measurements for the corresponding location at a plurality of times, e.g. whether an average of multiple (temporally spaced) temperature measurements at a location associated with a given container is above the temperature threshold. Temperature and power consumption may be monitored using sensors (e.g. temperature sensors, or current / voltage sensors) by low level hardware services. The shared resource monitor may receive the sensor output, and correlate excessive temperature, or power (or memory readings) to events happening at a particular time within, or due to the operation of, an isolated execution environment (container), for example as recorded in a log as described below.

[0105] Alternatively or additionally, the excessive use criterion corresponding to a given container may include at least one voltage test defined by a voltage threshold (or power test or current test, defined by a corresponding power or current threshold). For example, the excessive use criterion may comprise at least one voltage test of whether a measured voltage across a portion of the computer system associated with the isolated execution environment (e.g. a voltage across a processor employed by the isolated execution environment) is above a voltage threshold associated with a voltage test.

[0106] For example, the voltage of multiple portions of the computer system may be determined individually, and for each portion a corresponding voltage test may be performed of whether the corresponding measurement is above a corresponding voltage threshold (which need not be the same in all the portions). The excessive use criterion for any given isolated execution environment may then be based on whether the voltage tests for portion(s) of the computer system associated with the isolated execution environment are met.

[0107] Optionally, the voltage measurements may be recorded in a voltage log, and a given voltage test may be based on voltage measurements for the corresponding portion at a plurality of times, e.g. whether an average of multiple voltage measurements at the respective times is above the voltage threshold

[0108] The temperature and / or voltages measurement(s) may be performed repeatedly at intervals (e.g. at intervals of at least a second, at least 30 seconds, of at least once a minute). Optionally, the determination of whether the excessive use criteria are met can be performed whenever the measurements are made. As noted, some of the tests may be depend on a log of past operation of the isolated execution environment, e.g. including a temperature log or a voltage log. Alternatively or additionally, the log may directly record the usage of resources at different times. The log may record which of the containers used which resources (and how much of those resources), e.g. at times recorded in the log. For example, CPU intensive operations that could cause a system starvation may be recorded in the logs. Similarly, memory is shared resource, so the system may have a mechanism to correlate this with container operations, and may record this information in the log.

[0109] For example, the log may include any one or more of:

[0110] - a log of the amount of memory (e.g. memory 105 and / or 109) and / or CPU operations (e.g. operations of CPU 103) employed by each of the isolated execution environments;

[0111] - firewall and / or network traffic logs recording firewall and network traffic associated with each of the isolated execution environments;

[0112] - a log of requests by each of the isolated execution environments for performance of computer operations performed by an unit of the computer system or by another computer system. For example, the computer operations may be cryptographic operations, e.g. performed by the crypto co-processor 111 or by the main CPU 102.

[0113] For example, for a certain container, the corresponding excessive use criterion may be based on a set of multiple tests including a test of CPU usage. The test may, for example, be that CPU usage of the container, as recorded by the log, is greater than a threshold X (a numerical value) of operations / second for greater than Y (another numerical value) seconds. This test may be indicative of a Distributed Denial-of-Service (DDoS) attack. If it is determined that this CPU usage test is met for a certain container, the shared resource monitor may determine that the corresponding excessive use criterion is met for that container, and the corrective action is taken. That is, container is modified, with or without administrator involvement.

[0114] Optionally, the excessive use criterion may be the same for all the isolated execution environments, for example based on the same set of threshold values (though in the case of temperature tests and / or voltage / power / current tests, they would relate to different corresponding locations / portions of the HSM, which are associated with (e.g. used by) those containers).

[0115] Alternatively, the at least one excessive use criterion may be different for different ones of the isolated execution environments (e.g with a different set of threshold values for different ones of the containers). They may be set in any way.

[0116] For example, if is known in advance that a certain one of the programs requires a high amount of memory or CPU operations, then a corresponding memory usage test or CPU usage test included in the set of tests defining the corresponding excessive use criterion may be defined with a higher memory usage or CPU operations threshold. This may permit the corresponding isolated execution environment to execute the program while employing more memory or CPU operations than other isolated execution environments.

[0117] Alternatively or additionally, the excessive use criterion may be based on the identity of the user who transmitted the request to the HSM to execute the corresponding program. This information may be included in the request, e.g. as a token identifying the user.

[0118] The present concept may optionally be used a part of a system in which a user organizes with an administrator (e.g. by a process including making a payment to the administrator or an organization associated with an administrator) that isolated execution environments which will be initiated based on requests that user sends for a program to be executed will be given a higher permitted power usage and / or priority level, e.g. for a specified number of requests to execute a program, than the isolated execution environments which are initiated based on requests which at least one other user sends for a program to be executed. This may be implemented using a certificate which the computer system selects from a database of certificates, upon a user transmitting to the computer system a request to execute a program. The certificate is configuration information which defines an isolated execution environment, with different ones of the certificates being different configuration information, which may include the permitted power usage limit and / or priority level, and may include much more information. Thus, the configuration of an isolated execution environment can be tailored by the selection of one of the certificates, e.g. so that an isolated execution environment can be different for ones of the users and / or for requests of different types. This provides a convenient way of controlling selective access to the HSM, and optionally to other resources via the HSM. Conveniently, the excessive use criterion for a given isolated execution environment may also be defined based on the certificate. For example, a given certificate may contain data (a field) which is used by the computer system to configure the excessive use criterion for container(s) generated based on the certificate. This data may for example, specify any one or more (or all) of the threshold(s) based on which the corresponding excessive use criterion is based, either by including numerical values for those threshold(s) or by containing index values which the computer system can use to extract those threshold(s) from a look-up table, e.g. stored in the computer system.

[0119] In one option, one or more of the certificates stored in the certificate database may be associated with corresponding ones of a plurality of users (i.e. there is a mapping between certificates and users), and the selected certificate is selected as a certificate associated with the user from whom the request was received. Thus, requests from different users may conveniently be processed using differently configured isolated execution environments. The mapping may be defined by association data (e.g. stored in the certificate database). Note that under the mapping, a given one of the users may be associated with multiple ones of the certificates (as described below) and a given one of the certificates may be associated with multiple ones of the users.

[0120] For example, different users may be trusted to different respective levels. A user who can be trusted (e.g. not to provide program requests which present security risks) may be associated with certificate(s) which give the corresponding isolated execution environment greater freedom than the certificate(s) of another user.

[0121] In another example, different users may be associated with different professional roles (e.g. within an organization). A user who typically performs tasks which require greater access to the resources of the HSM may have certificate(s) which give the corresponding isolated execution environment greater freedom than the certificate(s) of a user who requires more limited access to the resources of the HSM.

[0122] Optionally, associations between users and certificates made, e.g. by an administrator of the system updating the association data, in response to a payment, e.g. a payment to an owner of the HSM by the user (or an organization with which the user is associated). Thus, the associations may at least in part reflect the sums which the users have paid.

[0123] In general terms, the certificates may define access to any resources of the computer system itself, or at any devices outside the computer system. In this way, the resources / devices may be shared between multiple users, according to proportions defined by the certificates associated with each of the user. Each user may be associated with a different corresponding isolated execution environment which is initialized, following a request from that user, based on a certificate associated with that user and extracted from the certificate database. The multiple isolated execution environments may run on the computer system in parallel, i.e. with operations of different ones of the execution environments being performed substantially simultaneously by a processor of the computer system which is capable of parallel processing, or with operations of the multiple isolated execution environments being interleaved.

[0124] For example, a certificate may indicate that the isolated execution environment may use a CPU (or other processor device) of the computer system but with an operation rate limit, e.g. specifying that the number of operations performed by the processor device in a given time (“unit time”) has an upper limit. Alternatively or additionally, it may specify a memory usage limit for a memory of the computer system and / or of a memory device external to the computer system.

[0125] The limit(s) may be fixed, or may be conditional. For example, they may depend upon whether other isolated execution environments are currently running on the computer system. For example, each certificate may be associated with a CPU usage priority value, and, at any given time, the operation rate limit for each isolated execution environment may be a function of the CPU priority values of all the isolated execution environments which have been initiated and have not yet been deactivated. In another example, each certificate may be associated with a memory usage priority value, and, at any given time, the memory usage limit for each isolated execution environment may be a function of the memory usage priority values of all the isolated execution environments which have been initiated and have not yet been deactivated.

[0126] The certificate may alternatively or additionally specify permitted system calls and / or permitted device calls which the isolated execution environment is permitted to make, to access functionality of the computer system or external device(s).

[0127] For example, the configuration information of each certificate may specify whether a hardware implementation of at least one of an encryption algorithm, a key exchange protocol and a signature algorithm can be accessed. Alternatively or additionally, the configuration information of each certificate may specify whether a software implementation of at least one of an encryption algorithm, a key exchange protocol and a signature algorithm can be accessed.

[0128] Typically, the request to execute the program defines one or more requirements for the operation. For example, it may define explicitly any one or more of a CPU usage limit (i.e. a maximum number of operations per unit time the program must be able to access), memory usage limit (i.e. a maximum amount of memory to which the program must have access), and / or system / device calls which must be available. Alternatively, these may be implicit from the program (e.g. the program may include certain device calls or include routines which will generate data of a certain size to be stored).

[0129] Optionally, e.g. before initializing the execution environment, the computer system may include a compatibility determination process for determining whether at least one requirement defined by the request is compatible with the certificate(s) in the certificate database associated with the user from whom the request was received. This option is particularly valuable in cases in which the operation of the isolated execution environment during the execution of the program is not monitored (see below). If the determination is negative, an error resolution process may be initiated, e.g. an error message may be transmitted to the user. In the case of a user associated with multiple certificates, the compatibility determination process may determine if any of those certificates is compatible with the at least one requirement, and only perform the error resolution process if none of the certificates is compatible with the at least one requirement. Optionally, the compatibility determination process may not be performed, e.g. it may initially be assumed that the requirements of the request are compatible with the certificate(s) associated with the user, at least until monitoring of the isolated execution environment shows that this is not the case.

[0130] As noted, optionally one or more of the users may be associated with multiple ones of the certificates, specifying different respective configuration information. Upon receiving a request from a user associated with multiple certificates, a certificate is selected from the multiple certificates associated with the user (e.g. from among one or more certificates associated with the user which are determined to be compatible with at least one requirement defined by the request). The isolated execution environment is then initiated based on the selected certificate. The selection of the certificate may be at least partly based on the request itself (e.g. so that the certificate selected is the one which is most appropriate to one or more requirements defined by the request). For example, the certificate may be selected (e.g. from among the certificates associated with the user who transmitted the request and which were determined to be compatible with at least one requirement defined by the request) as a configuration which, according to a certain selection criterion, is best suited to execute the program. For example, it may be selected as the certificate which exceeds a requirement defined by the request by the smallest amount.

[0131] Alternatively, the certificate may be specified explicitly by the request (e.g. by including a certificate index number labelling the certificate in the certificate database), or may be selected according to a predefined certificate preference of the user. For example, the user may define an order for the certificates, and the certificate may be selected as the first certificate in that order, e.g. the first certificate which is compatible with the at least one requirement defined by the request.

[0132] The operation of the initiated isolated execution environment to execute the program may be monitored by the shared resource monitor. Upon determining that system resources employed by an isolated execution environment exceed a corresponding excessive use criterion defined by a set of tests defined by respective thresholds, the computer system may take the corrective action, e.g. interrupt the execution of the program. Any of the thresholds may be one of the limits discussed above defined by the configuration file / certificate, or a slightly higher value, or a threshold which is a default value for the HSM.

[0133] According to another aspect, there is provided a carrier medium comprising computer readable code configured to cause a computer to perform any of the above methods.

[0134] According to another aspect, there is provided a non-transitory computer readable storage medium comprising program instructions stored thereon that are executable by a computer processor to perform any of the above described methods. The methods are computer-implemented methods. Since some methods in accordance with embodiments can be implemented by software, some embodiments encompass computer code provided to a general purpose computer on any suitable carrier medium. The carrier medium can comprise any storage medium such as a floppy disk, a CD ROM, a magnetic device or a programmable memory device, or any transient medium such as any signal e.g. an electrical, optical or microwave signal. The carrier medium may comprise a non- transitory computer readable storage medium. According to a further aspect, there is provided a carrier medium comprising computer readable code configured to cause a computer to perform any of the above described methods.

[0135] According to another aspect, there is provided a device, comprising one or more processors configured to: obtain configuration information for an isolated execution environment; obtain a program; initialise an isolated execution environment based on the configuration information; and execute the program in the isolated execution environment.

[0136] In one example, the device is a hardware security module (HSM). For example, the device may be a hardware security module device as illustrated by Fig. 1 and have the logical structure as illustrated in Fig. 2.

[0137] Fig. 4 shows schematically the operation of an HSM, such as an HSM 11 as explained above with reference to Fig. 1 , in an implementation of the present disclosure. The HSM employs a container launcher service 210 as explained above with reference to Figs. 2, 3A and 3B. However, in contrast to the comparative example of Fig. 2, the HSM 11 of Fig. 4 further includes a shared resource monitor 401.

[0138] The shared resource monitor 401 may be implemented in software as a separate service (i.e. not as part of the HSM firmware 204). The monitor 401 may have access to, and employ, system statistics and / or logs generated by the HSM 11. Note that in a conventional HSM 11 , it is known to provide such a log, and to collect system statistics, for example for provision to a system administrator. The HSM 11 may, in the present implementation, be configured to provide the system statistics and / or access to the log to the shared resources monitor 401 . The HSM 11 may be configured to provide system statistics to the resource monitor 401 independently from the logs. The system resource monitor 401 may have similar privileges and permissions to carry out operations as a system administrator.

[0139] Note that in principle, an alternative way for the system resource monitor 401 to obtain information about at least some of the resources requested by container(s) is to route requests for resources (e.g. crypto operations) through the system resource monitor 401 . While this is possible, it would typically slow down operation of the HSM 11 undesirably, so using the system statistics and / or logs is advantageous. The container launcher service 210 is operative to launch containers 402, 403 associated with a first tenant (“Tenant 1”), containers 404, 405 associated with a second tenant (“Tenant 2”), and containers 406, 407 associated with a third tenant (“Tenant 3”). The containers 402, 404, 406 are to run HSM firmware, and the containers 403, 405, 407 are to run user-defined programs. During the operation of all the containers, the resource usage of the containers is separately monitored by the shared resource monitor

[0140] 401 , which (e.g. at intervals) determines if each of the containers meets an excessive use criterion. This may be the same for all containers (e.g. pre-set for the HSM 11) or different for different containers (e.g. with one excessive use criterion for the containers

[0141] 402, 404, 406, and with another excessive user criterion for the containers 403, 405, 407. The excessive use criteria may be pre-set for the HSM, or may be defined, e.g. when the containers are initialized.

[0142] The shared resource monitor may use association data associating multiple ones of the containers. For example, the association data may associated groups of one or more of the containers with corresponding tenants. The association data allows the shared resource monitor to monitor and control collectively a plurality of containers, e.g. multiple containers associated with a given tenant. For example, there may a hierarchy of monitoring. That is, a first level of monitoring may include identifying any of the tenants as a “suspect tenant”, e.g. a tenant for whom at least one of the associated containers meets a corresponding excessive use criterion, or a tenant for whom multiple associated containers collectively meet an excessive use criterion. Alternatively, a suspect tenant may be identified in another way, e.g. the tenant may be indicated by data supplied to the HSM. For a given suspect tenant, the shared resource monitor 401 may monitor differently the container(s) associated with the suspect tenant. For example, in some cases, the shared resource monitor 401 may not monitor resource usage by containers until the associated tenant is identified as suspect. Alternatively, the shared resource monitor may monitor all operating containers, but upon a given tenant being identified as suspect, the corresponding excessive use criteria for the containers associated with the suspect tenant may be made changed, such as being more stringent (e.g. the threshold(s) defining the criteria may be modified, e.g. lowered).

[0143] Alternatively or additionally, the corrective action taken by the shared resource monitor if one of the containers of a given tenant is found to meet the corresponding excessive use criterion, may include modifying (e.g. automatically, or by sending an alert to the system administrator) the operation the other containers which according to the association data are associated with the container which is found to have met the corresponding excessive use criterion, e.g. other containers associated with the same tenant. This modification may include suspending, or reducing the use of resources by, the associated containers. Optionally, this may only happen if the tenant is identified, or has previously been identified, as a suspect tenant. The corrective action may be performed without (directly) modifying the operations of containers which are not associated, according to the association data, with the container(s) which were found to have met the corresponding excessive use criterion, e.g. containers associated with other tenants.

[0144] Fig. 5 shows a method 500 which an HSM may perform according to an example of the present disclosure.

[0145] In a first step 501 , the HSM receives, from one or more users (tenants), multiple requests to execute one or more programs in the manner described above. For example, a single user may transmit to the HSM 11 multiple requests. Each request is that a respective program is executed by the HSM 11 . Alternatively, multiple users may each transmit one or more corresponding requests, where again each request is that a respective program is executed by the HSM 11. As explained above, the request comprises a container image 201. Once received, the container image 201 is stored (e.g. by the launcher service 210) in the working memory of the HSM 11 . The container image 201 comprises program code corresponding to the programs to be run in the container. The container image 201 further comprises all libraries, supporting files and programs, and any language interpreter required to run these programs on the HSM 11 .

[0146] In step 502, the computer system initializes a plurality of isolated execution environments based on the multiple requests. Each isolated execution environment may be for executing the program(s) stipulated in one or more of the requests. In an example discussed extensively below (e.g. in relation to Fig. 9), there may be a container for each program. Alternatively, in other cases, a given container may be initiated as described above in relation to Fig. 3A, and there may be multiple programs (e.g. for different corresponding users) within a single containers.

[0147] For each container, the initialization is carried out by the process described above. For each request the launcher service 210 initiates (launches) and runs a container 203 by sending an instruction to the container engine 202 to initiate a container 203 according to the corresponding container image(s) 201 of the program(s) the container is to execute, and the configuration file 207.

[0148] In step 503, the HSM executes the programs by running them within the corresponding container 203, in the manner described above. When the program in the container 203 is running, the program may run without requiring access to files outside the container 203.

[0149] In step 504, the computer system (HSM) is arranged to make measurement(s) of one or more resources measures indicating resources of the computer system employed by the isolated execution environment.

[0150] In one example, step 504 may include measuring a temperature of at least one corresponding location of the computer system, and / or a power usage by, and / or current through, and / or a voltage across, one or more portions of the computer. The measurements are examples of values termed here “resource measures”. In one example of measuring a temperature, respective temperature measurements may be made for multiple locations within the computer system. In one example of measuring power, current or voltage, the individual power / current / voltage of portion(s) of the computer system used by each isolated execution environment may be measured.

[0151] For example, step 504 may be carried out at least partly by the board support processor 113 using the temperature sensor(s) 114 and / or power / voltage sensor(s) 115. This information may be provided to the shared resource monitor 401 , e.g. by storing the information in a log to which the shared resource monitor 401 has access.

[0152] The shared resource monitor 401 may additionally, or alternatively, have access to other information (that is, other resource measures) quantifying resource usage by the containers. This may be a direct indication of resource usage. For example, the shared resource monitor may be have access to a log of memory usage by the isolated execution environment, a log of CPU operation usage by the isolated execution environment, firewall and / or network traffic logs for the isolated execution environment, logs of requests by the isolated execution environment for performance of cryptographic operations, and / or logs of requests by the isolated execution environment for operations performed by a crypto API.

[0153] In step 505, the shared resource monitor 401 determines whether the resource measures (measured temperature(s) and / or power / voltage usage(s) and / or other resource usages) for each of the containers meet corresponding set of tests, and based on that determination the shared resource monitor 401 determines whether any of the containers meets a corresponding excessive use criterion.

[0154] For example, for a given container, each of the resource measures relating to that container (e.g. quantifying the temperature / current / voltage / power usage of a portion of the computer system used by the corresponding container, or directly quantifying resources used by the container) may be employed in a corresponding test. The test may be met if the corresponding resource measure is above a corresponding threshold. For example, the measured temperature(s) may be compared with thresholds associated with temperature limit(s), e.g. a single temperature limit for all locations or respective temperature limits for different locations, and for each location there may be test of whether the corresponding temperature measurement is above the (corresponding) temperature limit.

[0155] If the shared resource monitor 401 determines that a given container meets any of the corresponding tests based on which the corresponding excessive use criterion is defined, it may determine that the container meets the corresponding excessive use criterion. Alternatively, the determination that a given container meets the excessive use criterion may be a more sophisticated function of the results of the corresponding tests. For example, for a given container there may be multiple temperature tests defined by different respective temperature thresholds, and the shared resource monitor 401 may use an excessive use criterion for that container which is that there is excessive use (i) if the temperature at any location associated with the container is above the higher temperature threshold, or (ii) if the temperature at any location associated with the container is above the lower temperature threshold and the average memory resources used by the container during a certain period are above a given memory threshold.

[0156] Fig. 5 is drawn on the assumption that it is determined it in step 505 that the excessive use criterion is met for (at least) one of the isolated execution environments (if it were not met for any container, the method 500 would terminate after step 505). That isolated execution environment is identified in step 505.

[0157] Accordingly, in step 506 a corrective action is taken, by e.g. by the shared resource monitor 401. In this action, or as a result of it, a modification is made to the identified isolated execution environment(s). However, in contrast to the modification described above for the comparative example, in the step 506 the one (or more) identified isolated execution environments are modified without interrupting the operation (i.e. suspending or terminating) all the isolated execution environments. In particular, the operation of the isolated execution environments other than the identified one(s) is preferably not modified

[0158] In principle, the modification may be to interrupt the identified isolated execution environment(s), or to throttle it / them down (reduce its / their access to resources), e.g. reduce the number operations per second the identified isolated execution environment(s) are permitted to instruct the main processor 103 to perform, e.g. when requesting that the firmware performs crypto-operations.

[0159] The corrective action taken by the shared resource monitor 401 in step 506 may be a modification to the identified isolated execution environment(s) which is performed automatically, or may involve sending a message to a system administrator, to alert the system administrator to a need for the system administrator to enter a command to cause the modification.

[0160] As noted above, the corrective action 506 may include modifying the operation of any other containers associated with the same tenant. However, it may not include directly modifying the operation of containers other than the container(s) identified in step 505, and in particularly it may not include directly modifying the operation of containers associated with other tenants. For simplicity, the possibility that the corrective action 506 includes modifying the operation of containers other than the identified one will not be discussed further here, but it is to be understood that this possibility is generally applicable.

[0161] The HSM 11 may be configurable, e.g. by a system administer, to determine which corrective action is taken in step 506, e.g. such that a user (system administrator) can select the degree of authority of the shared resource monitor 401 . Such configuration may be performed as part of the HSM installation. Some users (e.g. system administrators) will want the modification to be performed automatically, e.g. so that it is performed as soon as possible. Other users will want the corrective action to be a notification (alarm) to a system administrator, and for the system administrator to decide, e.g. based on their own internal security policies, to modify the operation of the identified isolated execution environment(s), rather than letting the shared resource monitor 401 do it on its own. As noted, the shared resource monitor 401 of the HSM 11 may support an assignment to each of the containers of different excessive use criterion / criteria which are used in step 505, e.g. based on different thresholds for the corresponding set of tests. For example, Fig. 6 shows a case in which respective excessive use criteria corresponding to each of three containers include a test defined by a respective power usage threshold (an energy usage limit) which is a numerical value. That is, the excessive use criterion is met by a container which uses an amount of power above the respective power usage threshold. As shown “container 1” is assigned a “high” value for the permitted power usage threshold, “container 2” is assigned a lower “medium” value for the permitted power usage threshold, and “container 3” is assigned a “low” value for the permitted power usage threshold. Any criterion may be used to set the respective thresholds, e.g. based on the container image which is used to define the program to be executed in the container and / or an identify of a user associated with a program which is to be executed in the container (e.g. this identify may be specified by the container image or separately).

[0162] Fig. 7 shows the logical structure of an HSM 11 according to an example of the present disclosure. This can also be an HSM 11 with the physical structure shown in Fig. 1. Unlike the logical structure of Fig. 2, however, an HSM 11 with the logical structure of Fig. 7 comprises the shared resource monitor 401 and an excessive use criteria database 208 which contains, for each of the still-active containers, a definition of the corresponding excessive use criterion (e.g. a set of thresholds, and optionally a set of locations / portions of the HSM 11 which are associated with the container). An excessive use criterion (that is, a definition of the excessive use criterion) may be assigned to a container when the container is initiated, and the excessive use criterion may be stored in the database 208 (e.g. by the launcher service 210 or the HSM firmware 204).

[0163] For example, the HSM firmware 204 may process the container image 201 received with the request to execute a program, and determine from that the corresponding excessive use criterion.

[0164] In another alternative, the excessive use criterion may depend, at least partly, on the still-active containers which presently exist, e.g. on the number of such containers and / or upon the permitted power usage limits of those containers. For example, upon receiving a new request to execute a program, the HSM firmware 204 may estimate an amount of power a new container which is to execute the program would need to generate for the overall power generation of the HSM 11 to exceed its operating limits. It may then define an excessive use criterion for the new container based on a power usage threshold which is lower than this amount of power generation. The excessive use criterion is then stored in the excessive use criteria database 208.

[0165] The launcher service 210 of the HSM 11 initiates a corresponding isolated execution environment as explained above, within which the program is implemented.

[0166] As explained above with reference to Fig. 3A, in some cases the HSM 11 may initiate a container including multiple (e.g. two) programs. In some cases, each program may be associated with corresponding thresholds for each of a set of tests. In this case, the launcher service 210 or HSM firmware 204 may assign an excessive use criterion to the container based on the corresponding tests for each program, and using for each test a corresponding threshold (e.g. each temperature and power usage threshold) which is the highest out of the corresponding threshold for each of the programs of the container.

[0167] Turning to Fig. 8, the logical structure of a hardware security module (HSM) which is a further example of the present disclosure is shown. This HSM, like the one of Fig. 7, may have the physical structure shown in in Fig. 1. The logical structure is similar to the one shown in Fig. 7, except that it additionally includes a HSM certificate database 209 storing multiple certificates associated with corresponding ones of a plurality of users. The certificate database 209 may further store association data defining an association (mapping) between users and certificates, as described below with reference to Fig. 12. The certificate database 209 may be stored in the non-volatile memory 105 in Fig. 1.

[0168] Each certificate in the certificate database may be associated with a signature dataset generated from the certificate using a key. The signature dataset may be checked when the certificate is downloaded into the certificate database 208, i.e. a check is performed that the signature dataset meets a validity criterion. The validity criterion may comprise a requirement that the signature dataset was generated from the associated certificate using the key. The signature dataset may further specify a validity period, and the validity criterion may include a check that the validity period has not yet expired.

[0169] As described below, upon receiving a request from a user to execute a program, one of the certificates can be selected and extracted by the launcher service 210. Optionally, the launcher service 210 may check the signature dataset associated with the extracted certificate (i.e. according to the validity criterion). This check may include checking that the associated validity period has not expired. If the check is not successful, the launcher service may generate an error message, and the process may terminate.

[0170] If the check is successful (or if there is no check), the launcher service 210 forms one of the configuration files 207 from the extracted certificate, to initiate a container. The permitted power usage and / or priority of the container(s) can thus be specified by the corresponding container. The launcher service 201 or HSM firmware 204 may also define a corresponding excessive use criterion for the container. This may be stored in the excessive use criteria database 208, for use during the operation of the container by the shared resource monitor 401 by performing the method 500 of Fig. 5.

[0171] The virtualization stack for the HSM with the logical structure of Fig. 8 is shown in Fig. 9. Instead of the single container 203 of Fig. 3A, in the HSM 11 of Fig. 7 the visualization stack shows two containers 203a, 203b associated with two respective users (“user 1” and “user 2”). Although just two containers 203a, 203b for just two corresponding users are shown, there may be any number of containers corresponding to different users. In the case of a user who causes multiple requests to be transmitted to the HSM to execute programs, the requests may generate multiple containers which exist concurrently, e.g. according to corresponding ones of the certificates (e.g. the same certificate for all the containers, or different certificates for different ones of the containers).

[0172] The containers 203a, 203c exist concurrently (i.e. a period is shown after both containers 203a, 203b have been initiated and before either is terminated (deactivated or deleted)). Each container 203a, 203b contains a corresponding program (“user code 1” and “user code 2”) and supporting files for the program. Each of the containers 203a, 203b is initiated based on a corresponding configuration file / certificate, which may be a certificate associated with the corresponding user. The configuration files / certificates for different users may be different or the same.

[0173] As in Fig. 3A, the Operating System kernel 302 of Fig. 9 runs on the HSM hardware 301. The OS kernel 302 runs the container engine 202, the launcher service 210, and the HSM main application 204 (HSM firmware) as separate processes. In the examples described below, the Operating System (OS) is a Linux Operating system. The container engine 202 runs the containers 203a, 203b. The runtime environment of the containers 203a, 203 are isolated from each other and from the runtime environment of the HSM firmware 204, e.g. such that data cannot pass between them. Various known container engines may be used. Examples of container engines include Docker, Linux LXC and Linux LXD.

[0174] Returning to Fig. 8, a request from a first user (e.g. user 1) to the HSM 11 to execute a program comprises a container image 201. Once received, the container image 201 is stored (e.g. by the launcher service 210) in the non-volatile memory 105 of the HSM 11 . The container image 201 comprises program code corresponding to the programs to be run in the container. The container image 201 further comprises all libraries, supporting files and programs, and any language interpreter required to run these programs on the HSM 11.

[0175] In response to the request, the launcher service 210 selects a certificate from the certificate database 209 and extracts it. For example, the launcher service 210 may select the certificate (e.g. associated with of the first user according to the association data), as a certificate associated with the first user. The launcher service 210 stores the certificate as a configuration file 207a. It or the HSM firmware 204 also stores a corresponding excessive use criterion, e.g. based on the certificate, in the excessive use criteria database 208. Optionally, the launcher service may perform the check described above using an associated signature dataset; if so, it is assumed here that the check is met. The launcher service 210 initiates (launches) and runs a container 203a by sending an instruction to the container engine 202 to initiate a container 203a according to the container image 201 and a configuration file 207a. When the container 203a is running, the programs may run without requiring access to files outside the container 203a.

[0176] The same process is conducted when a second user (say user 2) transmits a request to the HSM 11 to execute a program. In this case, the launcher service 210 extracts a certificate (e.g. associated with the second user according to the association data), and stores it as a configuration file 207b. The launcher service 210 initiates (launches) and runs a container 203b by sending an instruction to the container engine 202 to initiate a container 203a according to the container image 201 and a configuration file 207b.

[0177] The configuration files 207a, 207b define one or more of: 1 ) how the network is virtualized in the corresponding container 203a, 203b (e.g. which of the host's network interfaces are accessible to the runtime environment of the container); 2) a system call filter (e.g. which controls the system calls that can be made by a program executing in the container); 3) the Linux Capabilities available to the program executing in the container (e.g. controlling whether access is provided to specific kernel resources that were previously unavailable to unprivileged processes); and 4) the hardware devices (e.g. a quantum entropy source for random number generation) mounted in the OS file system that can be accessed (e.g. for read or write) by the programs executing in the container 203a, 203b.

[0178] Fig. 10 shows a method performed by a system comprising a HSM 11 and a client device 117 according to the example of the disclosure. The client device 117 corresponds to a user of the HSM device 11 . Here we assume that it is user 1 , but the process would be equivalent for a different user. The client devices for other user(s) of the HSM device 11 are not shown. As has been described previously, the client device 11 may be a computing device such as a desktop or laptop computer, or mobile device for example. As described previously, the HSM 11 may be directly coupled to the client computing device 117. Alternatively, the HSM 11 is located in a host system which is separate to the client computing device 117 and communication performed through the host system. The HSM 11 may be a HSM as has been described in relation to Fig. 1 and Fig. 8 for example.

[0179] In step 1001 , the client 117 the client 117 transmits a request to the HSM 11 to execute a program. The request comprises a container image. In an alternative example, the request comprises a reference (e.g. an address in memory or a file name) associated with a container image stored on the HSM 11 .

[0180] In either case, the container image comprises a user-defined program. The container image further comprises all libraries, supporting files and programs, and any language interpreter required to run these programs on the HSM 11 . The user therefore provides a container image including the user’s program, any required files, and any binaries that that user’s program depends on. The container image comprising the user-defined program may be provided from the client device 117 to the HSM 11 using various mechanisms. For example, EP3952202 - A DEVICE AND A METHOD FOR PERFORMING A CRYPTOGRAPHIC ALGORITHM, the entire contents of which are incorporated by reference herein, describes an example method of providing computer program code embodying a desired algorithm in an input data packet.

[0181] The request may be communicated between the client 117 and the HSM 11 over a secure channel to mitigate against tampering (e.g. by encrypting the desired container configuration at the client 117 in a way that can be decrypted by the HSM 11). In an example, the first secure channel uses the SSH protocol. In step 1002, the launcher service 210 of the HSM 11 selects a certificate from a certificate database and extracts it. In one option, the HSM firmware 204 identifies user 1 from the request, and the launcher service 210 extracts from the certificate database a certificate associated with user 1. This process is described in more detail below, including a case in which there are multiple certificates associated with user 1. Optionally, the launcher service may perform the check described above using an associated signature dataset. It is assumed here that the check is met; if not, the method terminates. The certificate is container information which specifies a desired container configuration. The launcher services stores the container information as a configuration file 207a. It also stores a corresponding excessive use criterion, optionally based on the container information, in the excessive use criteria database 208 for use during the operation of the container by the shared resource monitor 401. In an example the information indicating the desired container configuration comprises one or more of: 1) devices of the HSM 11 that are to be made accessible to the container, 2) capabilities of the container (e.g., an indication of the root privileges available to a process executing in the container), 3) an indication of the system calls that are to be supported by the container (e.g. the OS system calls that can be made by the process in the container and actioned by the kernel), and / or 4) whether the container is to have internet connectivity.

[0182] In step 1003, the launcher system 210 initializes a container 203a that is configured according to desired container configuration obtained in step 502.

[0183] In step 1004, HSM 11 executes the program in the container image within the container 203a.

[0184] Fig. 11 A is a schematic illustration of a first manner of operation of an HSM 11 with the logical structure of Fig. 8. Upon a user transmitting to the HSM 11 a request to execute a program, the launcher service 210 accesses a certificate database 209 and extracts a certificate from the certificate database 209. Optionally, the launcher service may perform the check described above using an associated signature dataset. It is assumed here that the check is met; if not, the method terminates. The extracted certificate forms a configuration file. In the case that the request is from user 1 , the configuration file is configuration file 207a. The launcher service 210 initiates the corresponding container 203a based on the configuration file 207a. The container 203a executes the program while communicating with the container engine 202 and the HSM firmware 204. The communication with the HSM firmware may use an API defined by the HSM firmware. Fig. 11 B is a schematic illustration of a second manner operation of an HSM with the logical structure of Fig. 8. This differs only by how the container 203a communicates with the HSM firmware. Specifically, in the operation shown in Fig. 11 B the container 203a is provided with a proxy (proxy server) 1101 , an item of software which filters communications between the container 203s and other applications running on the operating system 302. In particular, the proxy server 1101 can be arranged to communicate with a process 602 executed by the HSM firmware 204. Communications between the container engine 202 and the container 203a can be via the proxy 1101 and the process 1102. The process 602 may be a “daemon”, a program executing as a background process.

[0185] Turning to Fig. 12, associations are shown between a plurality of users 1201 , 1202, 1203 and a plurality of certificates 1204 stored in the certificate database 209. In this example, the certificate database stores four certificates, labelled by certificate index numbers 1 to 4. The user 1201 is associated with the certificates labelled 1 and 2. The user 1202 is associated with a single certificate, which is the one labelled 3. The user 1203 is associated with certificates labelled 4 and 1. Note that both users 1201 and 1203 are associated with the certificate labelled 1.

[0186] The associations are defined by “association data” stored in a mapping table. The mapping table may be stored in the certificate database 209. The association data shows, for each of the users, which of the certificates 1204 each of the users is associated with. Thus, upon consulting the mapping table, the HSM 11 (e.g. the HSM firmware 204) can determine which certificate(s) are associated with a given user. For example, the HSM can determine that the user 1203 is associated with the certificates labelled 1 and 4.

[0187] The mapping table may further contain information which is used, upon a request being received from a user to execute a program, to select one of multiple certificates associated with the user. That is, the mapping table may indicate a predefined certificate preference of the user. For example, the mapping table may define an order for the certificates for a given user such that, when a request is received from that user, one of the certificates associated with that user can be selected as the certificate which is first in the order, and which optionally meets at least one additional criterion. As discussed below, the additional criterion may be that the certificate is compatible with at least one requirement defined by the request. In principle a user who is not associated with any of the certificates may be submit a request to execute a program. The certificate database 209 may store at least one default certificate for such a user. One of the default certificate(s) is selected when such a user submits a request to execute a program.

[0188] Fig. 13 shows a portion of Fig. 12 in more detail, and in particular shows the configuration information for the two certificates 1204a, 1204b (the ones with certificate index numbers 1 and 2) associated with user 1201. The certificate 1204a contains configuration information which defines an isolated execution environment which is allowed to access hardware accelerated operations of the HSM (i.e. operations which are implemented on the HSM 11 in hardware rather than software). Furthermore, the configuration information for the certificate 1204 has relatively high (i.e. higher than the certificate 1204b) values for one or more limits, including a CPU usage limit, a RAM usage limit and / or a permitted power usage limit. Instead of or in addition to the permitted power usage limit, the certificate 1204b may include a priority value indicating the priority of a container initiated based on the configuration information of the certificate. The priority of certificate 1204a is shown in Fig. 13 as being high.

[0189] The certificate 1204b contains configuration information which defines an isolated execution environment which is not allowed to access hardware accelerated operations of the HSM, but is allowed to access software-implemented algorithms performed by the HSM firmware 204. Furthermore, the configuration information for the certificate 1204 has relatively low (i.e. lower than the certificate 1204a) values for one or more limits, including a CPU usage limit, a RAM usage limit and / or a permitted power usage limit. Instead of or in addition to the power usage limit, the certificate 1204b may include a priority value indicating the priority of a container initiated based on the configuration information, and in Fig. 13 the priority for certificate 1204b is shown as being low.

[0190] Fig. 14A shows a method of defining a certificate 1204. The method may be performed by a supplier of the HSM 11 , or by an owner or administrator of the HSM 11 . It may be performed in the HSM 11 by transmitting commands to the HSM 11 (e.g. through the interface 107) but may alternatively be performed using a separate computer system, for example one having a graphical user interface and data entry devices. The method comprises performing one or more of step 1401 , step 1402, step 1403, and / or step 1404 in any order; and then performing step 1405 and optionally step 1406. As discussed above, in an example the container configuration information comprises one or more of: 1) devices of the HSM that are to be accessible to the programs executing in the container; 2) the Linux capabilities that are accessible to programs executing in the container; 3) the system calls that can be made by programs executing in the container and / or 4) whether the programs executing in the container have internet access.

[0191] In step 1401 , the method comprises defining (e.g. by manual data input) the system calls that are to be made available to the processes (e.g. the user-defined program) running in a container defined by the certificate. A system call is a programmatic way in which a computer program requests a service from the kernel of the operating system. All computer programs that use computer resources use system calls. System calls are used for process creation, process management, memory management, networking, device I / O handling etc. Linux system calls include: fork(), exit(), exec(), open(), read(), write(), close() etc.

[0192] In step 1402, the method comprises defining the Linus capabilities of processes accessible to programs executing in the container (e.g. by manual data). For example, the administrator can manually type the desired capabilities into a file containing the container configuration information. This may include setting one or more limits, including a CPU usage limit, a RAM usage limit and / or a permitted power usage limit. Instead of or in addition to the permitted power usage limit, step 1402 may include defining a priority value. Note that the permitted power usage limit may be set individually, or it may be implicit, i.e. a function of other limits set in step 1402, e.g. the CPU usage limit and / or the RAM usage limit.

[0193] In step 1403, the method comprises defining the devices of the HSM that are to be accessible to the container (e.g. by manual user input).

[0194] In step 1404, the method comprises defining the internet connectivity status of programs executing in the container. In an example the internet connectivity status comprises an indication of whether or not internet connection is permitted (e.g. Internet connection = yes / no). In this example, the supplier / administrator / owner has a binary option “internet connectivity: on / off”, where the latter specifies that program is run “offline” - in this case no network connectivity features are available. Steps 1401 , 1402, 1403, and 1404 are shown in parallel in Fig. 14A, and these steps may be performed independently of each other. However, as has been described previously, the steps may overlap. Furthermore, in some examples, one or more of steps 1401 , 1402, 1403 and 1404 are performed without all of steps 1401 , 1402, 1403 and 1404 being performed. In this case, the steps performed will determine the extent to which the configuration of the container on the HSM is controlled. For example, in the case where only step 1401 is performed, then the container functionality would only be limited (further than a default configuration) in terms of the system calls that can be made by the program executing in the container.

[0195] After performing one or more of steps 1401 , 1402, 1403 and 1404, the method proceeds to step 1405.

[0196] Step 1405 comprises generating container configuration information. In this example, step 405. Step 1405 comprises generating a file comprising a representation comprising the system calls, the capabilities, the devices, and / or the internet connectivity status (depending on which of steps 1401 - 1404 are performed). In an example generating the desired container configuration information comprises combining one or more of: the system calls defined in step 501 , the capabilities defined in step 1402, the devices defined in step 1403, and / or the internet connectivity status defined in step 1404 into a single list.

[0197] In step 1406, the configuration information is provided to the certificate database 209 as a certificate. For example, if the steps 1401-1405 are performed using a computer system external to the HSM 11 the configuration information may be transmitted to the certificate database 209 via the interface 107 and the HSM firmware 204. Note that in embodiments in which steps 1401-1405 are performed using the HSM 11 itself (e.g. defining the new certificate in the non-volatile memory 105) no separate step of transferring the configuration to the certificate database may be needed; that is, steps 1401-1405 may define the certificate within the certificate database 209.

[0198] Step 1406 typically comprises providing the certificate with a certificate index number which may be used to reference the certificate.

[0199] The process of Fig. 14A may be performed multiple times, to define multiple certificates within the HSM 11. For example, steps 1401-1405 may be performed using an external computer system to define multiple certificates, and then step 1406 may be performed for each of the certificates to upload it into the certificate database 209. Optionally, the method of Fig. 14A may be performed again later (e.g. after the HSM 11 has already executed some programs the process shown in Fig. 8) to increase the number of certificates in the certificate database 209.

[0200] Fig. 14B shows the steps of a method for defining the association data. This step may be performed after the method of Fig. 14A, though in principle it may be performed before it.

[0201] In a first step 1451 an administrator receives a request to associate a user with at least one of the certificates of the certificate database 209. This request may identify the certificate(s) with which the user wishes to become associated, e.g. using the corresponding certificate index number(s). The user may be an individual or organization which is not currently associated with one of the certificates (a “new user”) and wishes to become associated with one or more of the certificates, or a user who is already associated with one or more of the certificates (an “existing user”) and wishes to become associated with one or more certificates. In either case the new / existing user is typically aware (at least to some extent) of the capabilities of the certificate with which it requests to be associated. The request to associate the user with the certificate(s) may optionally include or reference a payment the new / existing user makes to the administrator or another organization.

[0202] Following a check that the new / existing user meets one or more conditions (e.g. relating to the identity of the new / existing user, and / or the payment being adequate), in step 1452 the administrator may determine to associate the new / existing user with the certificate specified in the request to associate the user with at least one of the certificates. The administrator does this by transmitting into the HSM 11 a command to modify the association data stored in the certificate database to associate the new / existing user with the certificate(s) requested.

[0203] Fig. 15 shows a process performed by the HSM using the certificates and the association data. That is, it shows the process of Fig. 10 from the point of view of the HSM 11 described previously for example with reference to Fig. 1 and Fig. 8. The HSM 11 stores program code (e.g. in non-transitory memory) that implements the steps 1501-1505 of Fig. 15 - for example, the method of Fig. 10 is performed by the HSM firmware 204 and the launcher service 210. In step 1501 of the method of Fig. 15, the method comprises receiving from a client device 117 (i.e. from a user) a request to execute a user-defined program. The request comprises the container image comprising the user-defined program and further comprises all libraries, supporting files and programs, and any language interpreter required to run these programs on the HSM 11. It further identifies the user who transmitted the request. It may do this by including a user token (e.g. an encrypted token) which the HSM 11 may compare to a database of user tokens.

[0204] In step 1502, upon receiving the request, the HSM 11 selects one of the certificates. Specific methods for conducting this step are explained below with reference to Figs 10 and 11.

[0205] In step 1503 the selected certificate is extracted from the certificate database 209 and passed to the launcher service 210. At this stage, the launcher system 210 may define a corresponding excessive use criterion, e.g. based on the extracted certificate, and store it in the excessive use criteria database 208 for use by the shared resource monitor 401 while the container is running.

[0206] In step 1504 a container is initialized, by the container engine 202 under the control of the launcher server 210, according to the container configuration information of the selected certificate extracted in step 1503.

[0207] Initializing the container comprises the launcher service 210 generating a container configuration file that includes the system calls, devices and / or capabilities specified in the selected certificate. Without loss of generality, we assume that the user is the “user 1”, so that the container is denoted 203a (as in Fig. 8 and Fig. 9) and the configuration file is denoted 207a. In an example, the generated container configuration file 207a comprises only the system calls, devices and / or capabilities specified in the container configuration information defined by the selected certificate (optionally as well as other configuration information). Generating a container configuration file 207a may comprise modifying a template container configuration file for example. The template container configuration file may be a default container configuration file. The launcher service 210 may store the configuration file 207a, e.g. in the working memory.

[0208] For example, the launcher service 210 may generate a seccomp profile that specifies the allowed system calls. The launcher service 210 includes the seccomp profile in the container configuration file 207a. This may involve modifying an LXC container configuration file to use the generated seccomp profile. The LXC container configuration file may also be modified to also only allow the enabled devices specified in the custom configuration information. For example, this may involve using cgroups - a Linux kernel feature. The LXC container configuration file may also be modified to according to the requested network configuration information.

[0209] The configuration file 207a is then provided to the container engine 202 (e.g. the container engine 202 is instructed by the launcher server 210 to read the configuration file 207a from the working memory). The syscalls, capabilities, devices and network availability are managed by the LXC container engine, where the LXC container engine calls the right functions from the OS kernel 302 to set the container up. For example, the container engine 202 generates a seccomp filter according to the seccomp profile in the configuration file 207a.

[0210] Once initialised according to the configuration, in step 1505 the user program is executed in the container with the allowed syscalls, capabilities and devices, e.g. in a conventional manner. This is based on the container image 201 for the corresponding program.

[0211] The execution of the container for the program is monitored by the shared resource monitor 401. This ensures that corrective action is taken if any of the currently-running containers meet the corresponding excessive usage criteria stored in the excessive use criteria database 208. For example, if the number of CPU operations (e.g. per unit time) or memory usage of a given isolated execution environment is determined to exceed a threshold defined by the excessive use criterion, the execution of the isolated execution environment may be automatically modified (interrupted and / or throttled down), and / or an error message may be sent to the client device 117, e.g. for display by the client device 117, so that an administrator operating the client device 117 may issue a command to modify (interrupt and / or modify) the isolated execution environment.

[0212] It will be understood that defining the container according to one of the certificates 1204 increases the system security. For example, restricting the system calls that can be made by a user-defined program within the container to be one of the certificates 1204 reduces the attack surface with which a user-defined program can attack the kernel and exploit the data (e.g. cryptographic keys) and the operation of the HSM 11. In particular, the attack surface is reduced when the container configuration is restricted to only those system calls that are required by a user-defined program because the number of permitted system calls is reduced and the ability of a malicious party to launch an attack on the kernel from within the container (e.g. by replacing the user-defined program with a malicious program) is mitigated since the container will only allow system calls that are used by the user-defined program.

[0213] Fig. 16 shows a first way of implementing step 1502 the method of Fig. 10. In an example the method of Fig. 16 is implemented by a HSM 11 , e.g. as shown in Fig. 1 and Fig. 8.

[0214] In step 1601 the method comprises identifying, using the association data stored in the certificate database, one or more certificates stored in the certificate database 209 associated with the user who transmitted the request. In the case of user who is not associated with any certificates, at least one default certificate may be selected.

[0215] If multiple certificates in the certificate database 209 are associated with the user who transmitted the request (or if, in the case of a user who is not associated with any certificates, there are multiple default certificates), the method of Fig. 16 comprises a step 1602 of selecting one of those certificates. There are several ways this could be done.

[0216] In one option, the selection may be based on at least one requirement defined by the request for the program to be executed. For example, the request many specify a certain maximum number of transactions per second is performed (e.g. by the CPU 103, or by another device). For example, the request may define a CPU usage limit. This may be defined explicitly in the request or be implicit from properties of the program. A certificate may be selected which specifies a CPU usage limit which is greater than the CPU usage limit specified by the request by the least amount (i.e. the selected certificate may be selected such that the CPU usage limit specified by the selected certificate minus the CPU usage limit specified by the request, is positive but as small as possible). In this way, a container is chosen which gives the least risk of an unacceptable amount of CPU being used, either by the user-defined code malfunctioning or if the user-defined code has been compromised by a malicious third party. If the request does not specify a CPU usage limit then a certificate may be selected which also does not, or which has a high CPU usage limit.

[0217] Although this example is based on selecting a certificate based on CPU usage limits, in variations the selection may be based on a memory usage limit, or a combination of multiple usage limits. Alternatively, the selection in step 1602 could be done based on data (e.g. included in the mapping table or included in the request) indicating a predefined certificate preference of the user. This may indicate an order for the certificates. Step 1602 may then include selecting the certificates as the first certificate in the preferred order defined by the data, out of the certificates identified in step 1601.

[0218] Note that in a variation, step 1601 could be omitted, and step 1602 could be performed in respect of all the certificates in the certificate database 209. In this variation the association data may not be needed and the mapping shown in Fig. 12 would not exist. This would still have the advantage that the most appropriate certificate for the job is selected, potentially leading to greater efficiency (e.g. by selecting a container configuration which has sufficient flexibility to implement the request) while not having unnecessary flexibility (leading to a greater security risk). However, it would mean that that selected certificate is not generally associated with the user who transmitted the request, so the selection of container capabilities according to which user transmitted the request would not be present.

[0219] Fig. 17 shows an alternative way of implementing step 1502 of the method of Fig. 15. Steps 1701 and 1704 correspond to, and are identical to, steps 1601 and 1602 of the method of Fig. 11 respectively. However, the method of Fig. 17 contains additional steps 1702 and 1703.

[0220] In step 1702 the method comprises a compatibility determination process for determining whether any of the certificate(s) determined in step 1701 to be associated with the user who transmitted the request for the program to be executed is compatible with at least one requirement defined by the request. For example, the configuration file may be examined to determine whether 1) the devices of the HSM that are required to be accessible to the program; 2) the Linux capabilities that are required to be accessible to the program; 3) the system calls that are required to be available to the program and / or 4) the program requires internet access. These requirement(s) are compared with the certificates determined in step 1602, to see if any matches.

[0221] If it is determined in step 1702 that none of the certificate(s) associated with the user from whom the request was received is compatible with the requirement(s) of the request, in step 1703 an error resolution process is performed. For example, the HSM 11 may transmit an error message to the client system 117, informing the client 117 that the request to execute the program is unsuccessful. The method of Fig. 17 may then end (i.e. the program is not executed). In this case the user may take action, e.g. by modifying the program or by transmitting a request to an administrator for the user to become associated with another certificate (e.g. one which places fewer constraints on the isolated execution environment it defines). Alternatively, the error resolution process 1704 may comprise modifying the configuration file to be compatible with one of the constraints. For example, if the requirement is that the program is permitted to control the CPU 103 to perform a certain number of operations in unit time, and that requirement exceeds CPU usage limit(s) specified by the certificate(s) identified in step 1701 , the configuration file may be modified so that the program can be run, but more slowly. In this case, the method of Fig. 17 may be performed again, and the second time the method may advance to step 1704 (that is, if multiple certificates were determined to be compatible in step 1702, determining which certificate is most appropriate for the job) using the modified configuration file. Note that in the case that step 1704 is performed by reference to requirement(s) of the program, as explained above with reference to step 1602, the requirements used in step 1704 may be different from those used in step 1702, e.g. fewer requirements may be used in step 1704 than in step 1702.

[0222] As explained above, although it is preferable for users to be associated with certificates as shown in Fig. 12, this is not essential. In this case, the methods of Figs. 16 and 17 may omit steps 1601 and 1701 respectively. That is, steps 1602 and 1702-1704 can be performed for all certificates in the certificate database 209.

[0223] Fig. 18 shows a method of dynamically updating a container configuration according to an example.

[0224] In step 1801 a certificate stored in the certificate database 209 is modified. This may be done because a vulnerability of container configurations according to the certificate has been identified. In some cases it may become apparent that one or more configuration settings (e.g. system calls, devices, capabilities etc.) that was once considered safe and therefore included in one of the certificates 1204, is no longer safe, for example, due to a recently identified vulnerability (e.g. a zero-day attack).

[0225] The modification may be done, for example, by an administrator or supplier of the HSM 11 transmitting a replacement version of the certificate to the certificate database 209 via the interface 107 and the HSM firmware 209. Alternatively, the administrator / supplier may transmit instructions (e.g. via the interface 107) to the HSM firmware to modify a certificate in the certificate database 209. Alternatively or additionally, step 1801 may include modifying one or more limits defined by the certificate, including a CPU usage limit, a RAM usage limit and / or a power usage limit, and / or modifying a priority value defined by the certificate.

[0226] In step 1802 it is determined by the HSM 11 whether any container (isolated execution environment) has already been launched based on the certificate before it was modified, and is still in existence (i.e. the program executing in the container has not yet terminated). The launch of the container may have happened days, months or years before step 1801 was performed.

[0227] In step 1803, the configuration of the isolated execution environment for any containers identified in steps 1802 is modified.

[0228] In a first approach, modifying the current configuration of the container comprises identifying the configuration parameter to be changed and then updating the configuration of the container instance. For example, this may involve using the "ixc config device remove" command to remove a device which was once permitted by the certificate, but now is no longer permitted.

[0229] In a second approach, the container instance is paused, the configuration of the container is updated and the container operation is then resumed. In another example the state of the processes in the container is saved, the container is then stopped, and a new container is generated with the saved state and the updated configuration settings.

[0230] In some cases, the act of changing the configuration of the container to remove the access to the vulnerable configuration may mean that, at some point, the user-defined program being executed in the container will make an attempt to access the vulnerable resource (e.g. by attempting to access the vulnerable device, or by attempting to use the vulnerable system call). In this case, the container will block such a request and an error will be generated, preventing a user-defined program from potentially exploiting a resource that is known to be vulnerable.

[0231] Alternatively, upon determining that the already-initialised container has a configuration that allows access to the vulnerable parameter (e.g. allows access to the vulnerable device or allows the vulnerable system call to be made), and upon determining that the desired container configuration was based on the program being executed in the container, then the container is stopped and removed. In this case, the program being executed in the container is no longer suitable for being executed on the HSM (because it requires access to a vulnerable command / device) and so the container and the program executing in the container is stopped, instead of waiting for an error as in the example above.

[0232] Optionally, at least one excessive use criterion associated with the container may be modified also (e.g. when the existing at least one excessive use criterion was defined based on data in the certificate).

[0233] The method of Fig. 18 reduces the attack surface available to a user-defined program and enables a more secure HSM 11.

[0234] In the examples above, reference is made to a user-defined program, however any program that is intended to be run in the container can be used in the methods described herein, whether or not it is defined by the user that wants to run it on the HSM 11 .

[0235] In the description above, the techniques are described in relation to a HSM 11 running a Linux Operating System (OS). However, the same techniques can be applied to other Operating Systems.

[0236] In the examples above reference is made to a container. As discussed above, a container is a standalone unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A container is an example of a sandboxed environment or an isolated environment for execution of a program. Other examples of isolated environments may be used however, for example a virtual machine.

[0237] The above described methods include customized restriction of container-based applications, or applications based in other isolated environments. There may be instances where assurance that a user-defined program is being executed in accordance with a regulatory or compliance regime is required. By providing custom configuration settings, a user may have assurance that the program is being executed in accordance with a regulatory requirement. There may be instances where this list is not appropriate however, especially where there are different tenants on the HSM that are operating under different regulatory requirements. By allowing restriction of the configuration of a container based on its use case, for example the regulatory regime it is operating under and the system calls used by the program it is executing, compliance with regulatory requirements may be provided and the attack surface reduced in a scalable way. In the above described methods, the container is customized according to its use requirements: the launcher service initialises the container according to a configuration file defined by a selected certificate, and the user-defined program is executed within the container. This may provide a secure implementation, with a reduced attack surface, i.e. a reduced opportunity to exploit a vulnerability of the HSM, while executing user code. The policy can also be reconfigured to respond to new threats. The method provides the ability to launch customised containers that restrict the system calls, network devices or capabilities of the container. A user can therefore ensure that the program is being executed in accordance with a regulatory requirement. This reduces the attack surface that a malicious user-defined program could exploit, for example.

[0238] Even removing a single kernel system call may provide improved security. For example, profiling the user-defined program may indicate that it does not use a first system call. The container is therefore initialised with a seccomp profile that does not include the first system call. If the program is compromised, an attacker may include functionality in the program that calls the first system call. If executed, this functionality may cause a catastrophic failure to the entire HSM. Reducing the container configuration to remove the first system call thus avoids a full compromise of the HSM, and may result in a more contained compromise of just the user’s program for example. The container is a possible source of attacks against the HSM - for example, an attacker might compromise the container to then extract sensitive data, e.g. keys, from the HSM. By controlling the configuration settings of the container, the HSM may be protected from the container.

[0239] The methods provide a safe secure execution environment (SEE) using a trusted platform, for example a HSM. In particular, the SEE is isolated from impacting the security functionality of the HSM's trusted computing base. However, the application still benefits from operating within the hardened environment of the HSM.

[0240] It will be understood that the invention is not limited to the embodiments described above and various modifications and improvements can be made without departing from the concepts described herein. Except where mutually exclusive, any of the features may be employed separately or in combination with any other features and the disclosure extends to and includes all combinations and sub-combinations of one or more features described herein.

Claims

CLAIMS:

1. A computer-implemented method comprising: receiving, by a computer system, from one or more users one more requests to execute one or more programs; initializing, by the computer system, a plurality of isolated execution environments; executing, by the computer system, the one or more programs concurrently in the isolated execution environments; determining for each of the isolated execution environments one or more corresponding resource measures indicating one or more resources of the computer system employed by the isolated execution environment; identifying any of the isolated execution environments for which the one or more corresponding determined resource measures meet a corresponding excessive use criterion; and upon identifying any of the isolated execution environments for which for which the one or more corresponding determined resource measures meet the corresponding excessive use criterion, performing a corrective action in respect of the identified isolated execution environment without interrupting the operation of all the isolated execution environments.

2. The method according to claim 1 in which the corrective action comprises transmitting a message indicating the identified program to an administrator.

3. The method according to claim 1 or claim 2 in which the corrective action comprises modifying the operation of the identified isolated execution environment without interrupting the operation of all the isolated execution environments.

4. The method of claim 3 in which modifying the operation of the identified isolated execution environment comprises interrupting the identified isolated execution environment.

5. The method of any preceding claim, in which at least one said resource measure for each isolated execution environment is determined by examining a log which is a record of past operation of the isolated execution environment.

6. The method of claim 5 in which the log is a record of the usage of the one or more resources by the isolated execution environment.

7. The method of claim 6 in which the corresponding excessive use criterion for a certain one of the isolated execution environments comprises a test of whether the log indicates that the usage of the one or more resources by the isolated execution environment is above a threshold.

8. The method of claim 7 in which the usage of the resource by the isolated execution environment measures a cumulative usage of the one or more resources during a specified period of time.

9. The method of claim 7 or claim 8, in which the log comprises a firewall log indicating interactions of the isolated execution environment with a firewall of the computer system.

10. The method of any of claims 7 to 9, in which the log comprises a network traffic log recording network traffic to or from the isolated execution environment.

11. The method of any of claims 7 to 10, in which the log comprises a log of requests by the isolated computer environment for the computer system to perform cryptographic operations.

12. The method of any of claims 7 to 11 , in which the log comprises a log of requests by the isolated computer environment for a cryptographic API of the computer system to perform an operation.

13. The method of claim any preceding claim, in which determining whether the excessive use criterion for a certain one of the isolated execution environments is met comprises at least one temperature test of whether at least one temperature measurement for a portion of the computer system is above a temperature threshold.

14. The method of claim 13 when dependent upon claim 5, in which there are a plurality of said temperature measurements recorded in the log.

15. The method of claim any preceding claim, in which the excessive use criterion for a certain one of the isolated execution environments comprises a test of whether at least one voltage measurement for a portion of the computer system is above a voltage threshold.

16. The method of claim 15 when dependent upon claim 5, in which there are a plurality of said voltage measurements recorded in the log.

17. The method of any preceding claim, in which the corresponding excessive use criteria of at least two of the isolated execution environments are different.

18. The method of any preceding claim, further comprising, upon receiving the requests for a program to be executed, the computer system, for each request: selecting a certificate from a certificate database which stores a plurality of certificates, each certificate being configuration information for defining an isolated execution environment and different ones of the certificates specifying different permitted power usages; and extracting the selected certificate; the initializing of each isolated execution environment being performed based on the configuration information of the extracted certificate.

19. The method of claim 18 in which the excessive use criterion for a given one of the isolated execution environments depends upon the corresponding configuration information.

20. The method of claim 18 or 19, wherein the configuration information of each of the certificates specifies at least one of: a CPU operation rate limit, a memory usage limit, permitted system calls, a permitted power usage limit, a priority value, or permitted device calls.21 . The method of claim 20 in which at least one of the permitted power usage limit or priority value for each isolated execution environment is dependent upon the identity of the user who sent the corresponding request for a program to be executed.

22. The method of any preceding claim in which different ones of the programs are associated with different corresponding ones of a plurality of tenants of the computer system.

23. A carrier medium comprising computer readable code configured to cause a computer system to perform the method of any preceding claim.

24. A device, comprising one or more processors configured to: receive from one or more users one more requests to execute one or more programs; initialize a plurality of isolated execution environments; execute the one or more programs concurrently in the isolated execution environments; determine for each of the isolated execution environments one or more corresponding resource measures indicating one or more resources of the computer system employed by the isolated execution environment; identifying any of the isolated execution environments for which the one or more corresponding determined resource measures meet a corresponding excessive use criterion; and upon identifying any of the isolated execution environments for which the one or more corresponding determined resource measures meet a corresponding excessive use criterion, perform a corrective action in respect of the identified isolated execution environment without interrupting the operation of all the isolated execution environments.

25. The device of claim 24, in which the device is a hardware security module device.

Images