Trusted Out-of-Band Memory Acquisition for IOMMU-Based Computer Systems

Abstract

Embodiments of the present disclosure generally relate to trusted out-of-band memory acquisition for an IOMMU-based computer system. An apparatus includes an interface and a memory acquisition circuit. The interface is configured to communicate over a bus operating according to a bus protocol, the bus protocol supporting address translation transactions that translate between bus addresses in an address space of the bus and physical memory addresses in an address space of a memory. The memory acquisition circuit is configured to read data from the memory by issuing one or more requests on the bus using the bus protocol, the one or more requests (i) specifying an address to be read in terms of the physical memory address, and (ii) indicating that the physical memory address in the request has been translated from a corresponding bus address, even though the address has not been obtained through any address translation transaction on the bus.

Description

Technical Field

[0001] The present invention relates generally to computer systems, and more particularly to a method and system for memory acquisition in an Input-Output Memory Management Unit (IOMMU) based computer system. Background of the Invention

[0002] In some computer systems, the central processing unit (CPU) communicates with various peripheral devices via a high-speed Peripheral Component Interconnect Express (PCIe) bus. PCIe is described in, for example, PCI Express 4.0, Version 0.9, Rev. 4.0, May 31, 2017, which is incorporated herein by reference. Chapter 10 of this specification defines the Address Translation Service (ATS). ATS provides translation services using a request-completion protocol between PCIe devices and the PCIe root complex (RC). The Address Type (AT) field is defined in the memory read and memory write transport layer packets (TLP). The AT field enables the root complex to determine whether a given request has been translated via the ATS protocol. Summary of the invention

[0003] Embodiments of the invention described herein provide an apparatus comprising an interface and memory acquisition circuitry. The interface is configured to communicate over a bus operating according to a bus protocol, the bus protocol supporting address translation transactions that translate between bus addresses in an address space of the bus and physical memory addresses in an address space of a memory. The memory acquisition circuitry is configured to read data from the memory by issuing one or more requests on the bus using the bus protocol, the one or more requests (i) specifying an address to be read in terms of the physical memory address, and (ii) indicating that the physical memory address in the request has been translated from a corresponding bus address, even though the address has not been obtained through any address translation transaction on the bus.

[0004] In some embodiments, by indicating that the physical memory address in the request has been translated, the memory acquisition circuit is configured to prevent software-controlled address translation from being performed as part of servicing the request. In some embodiments, by indicating that the physical memory address in the request has been translated, the memory acquisition circuit is configured to ensure that the request will be serviced entirely in hardware. In some embodiments, the memory acquisition circuit is configured to detect malicious operation by detecting that one or more of the requests have been denied.

[0005] According to an embodiment of the present invention, there is also provided a computer system comprising a memory, an input-output memory management unit (IOMMU) and a peripheral device. The memory can be accessed through a bus interface according to a bus protocol. The IOMMU is configured to connect a peripheral device to the memory through the bus interface, including performing an address conversion transaction for converting between a bus address in the address space of the bus and a physical memory address in the address space of the memory. The peripheral device is configured to read data from the memory by issuing one or more requests on the bus using the bus protocol, wherein the one or more requests (i) specify the address to be read according to the physical memory address, and (ii) indicate that the physical memory address in the request has been converted from the corresponding bus address, although the address has not been obtained through any address conversion transaction on the bus.

[0006] According to an embodiment of the present invention, a method is also provided, comprising: reading data from a memory by issuing one or more requests on a bus, the bus operating according to a bus protocol that supports address translation transactions for converting between a bus address in an address space of the bus and a physical memory address in an address space of the memory. The request (i) specifies an address to be read based on the physical memory address, and (ii) indicates that the physical memory address in the request has been translated from a corresponding bus address, although the address has not been obtained through any address translation transaction on the bus. Initiating a response action in response to detecting that one or more of the requests have been denied.

[0007] A more complete understanding of the present invention will be obtained from the following detailed description of the embodiments of the present invention in conjunction with the accompanying drawings; in the accompanying drawings: BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram schematically illustrating a tamper-resistant memory acquisition device in a computer system according to an embodiment of the present invention; and

[0009] Figure 2is a flow chart schematically illustrating a method for tamper-resistant memory acquisition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0010] Overview

[0011] Embodiments of the invention described herein provide techniques for accessing host memory via a bus by a peripheral device in a computer system in a tamper-resistant manner. An example use case is malware detection, but the disclosed techniques are applicable to any system or application requiring trusted, tamper-resistant memory access.

[0012] Embodiments described herein are generally related to computer systems based on a high-speed peripheral component interconnect (PCIe) bus and supporting address translation services (ATS). In some embodiments, a PCIe-based computer system includes a central processing unit (CPU), a host memory, and one or more PCIe devices. The computer system also includes an input-output memory management unit (IOMMU), which connects the PCIe device to the host memory.

[0013] Among other tasks, the IOMMU performs address translation from bus addresses to physical memory addresses. In this context, the term "physical memory address" refers to a physical address assigned in the physical memory of the host memory and corresponds to a corresponding physical storage location in the host memory. The term "bus address" refers to an address assigned in the address space of the PCIe bus. Bus addresses are sometimes also called "PCIe addresses" or "device addresses."

[0014] The ATS protocol specifies a negotiation phase in which a PCIe device requests a bus address to be translated into a physical memory address. The translation is performed by a Translation Agent (TA), which in turn queries the Address Translation and Protection Table (ATPT). After the address translation transaction, the PCIe device is able to issue memory requests (e.g., read requests and write requests) using the physical memory address.

[0015] While IOMMU and TA are typically implemented in hardware, the mapping between bus addresses and physical memory addresses in ATPT is typically configured through software, for example, by an operating system or hypervisor running on the CPU. Such software control is a security risk for memory acquisition solutions implemented using PCIe devices because tampering with the software that controls the mapping can result in erroneous and untrusted memory acquisition.

[0016] For example, consider a security tool connected to a PCIe bus as (or via) a PCIe device, and attempting to detect malware by reading data from host memory and analyzing the read data. If the tool uses the specified PCIe protocol, the malware can evade detection by tampering with the ATPT. For example, by modifying the address mapping in the ATPT, the malware can trick the security tool into thinking that it is reading a specific memory area in the host memory, when in fact it is given access to a different memory area. By using such techniques, the malware can create one or more memory areas that are invisible to the security tool.

[0017] In some embodiments of the present invention, a memory acquisition device is implemented in a manner that overcomes the above risks. The memory acquisition device includes an interface for communicating over a PCIe bus, and a memory acquisition circuit that reads data from a host memory by issuing a read request via the interface. The memory acquisition device issues a read request that uses the ATS protocol in a novel and unexpected manner.

[0018] In the disclosed embodiment, the memory acquisition device issues a read request that appears to use a previously translated address, but does not perform the prior negotiation phase (i.e., address translation transaction) defined in the ATS protocol. The IOMMU will normally perform such a read request in the host memory without performing the address translation because it incorrectly assumes that the address has already been translated. Since the ATPT is not consulted at all during this process, the address being read cannot be tampered with by manipulating the ATPT.

[0019] In some embodiments, the memory acquisition device issues a read request that appears to use a previously translated address by (i) specifying the address to be read as a physical memory address, and (ii) indicating that the address has been translated. As described above, there is no address translation request prior to the read request. In one embodiment, each read request includes a two-bit "Address Type" (AT) field. For example, see Table 10-1 in Section 10.2.1 of the PCIe Base Specification referenced above. The memory acquisition device sets the AT field to '10b' (defined as "translated") even though the physical memory address was not obtained during any address translation process.

[0020] When using the techniques described herein, the memory acquisition device causes the IOMMU to handle read requests entirely through unhackable hardware. All queries to the software-configurable ATPT are bypassed. As a result, the memory acquisition device is able to read regions of physical memory addresses of host memory in a trusted manner. Any tampering with the memory acquisition process, for example, by malware or due to misconfiguration or software failure, can be easily detected by the memory acquisition device. The techniques described herein enable dumping and analysis of both user space and kernel space memory regions.

[0021] System Description

[0022] Figure 1 2 is a block diagram schematically illustrating a computer system including a tamper-resistant memory acquisition device 24 according to an embodiment of the present invention. The computer system 20 may include, for example, a server, a workstation, a personal computer, or any other suitable computing platform.

[0023] In this example, system 20 includes a central processing unit (CPU) 28, a host memory 32, a memory acquisition device 24, and optionally one or more other peripheral component interconnect (PCIe) devices 36. Memory 32 typically includes one or more suitable random access memory (RAM) devices. PCIe device 36 may include, for example, a graphics processing unit (GPU), a network interface controller (NIC), or any other suitable peripheral device. Device 24 performs memory acquisition from host memory 32 using a method described in detail below.

[0024] The various elements of system 20 communicate via a PCIe bus. Each PCIe device (including device 36 and device 24) is connected to a PCIe root complex (RC) 40 via a corresponding root port (RP) 38. RC 40 is also connected to CPU 28 and memory 32.

[0025] In some embodiments, the system 20 also includes an input-output memory management unit (IOMMU—not explicitly shown in the figure). Among other tasks, the IOMMU supports address translation services (ATS). To this end, the IOMMU includes a translation agent (TA) 44 and an address translation and protection table (ATPT) 48.

[0026] TA 44 is typically implemented in hardware. Table 48 stores a mapping between bus addresses (in the address space of the PCIe bus) and corresponding physical memory addresses (in the physical address space of memory 32). Table 48 may be stored, for example, in a designated memory area in memory 32. Table 48 is configured by software running on CPU 28, for example, by an operating system (OS) or a hypervisor.

[0027] The agent 44 performs the conversion between the bus address and the corresponding physical memory address in response to the request from the PCIe device 36. After the conversion, the PCIe device issues a read request and a write request for the specified physical memory address. The ATS protocol is specified in detail in Chapter 10 of the PCIe Base Specification cited above. In addition to address conversion, the agent 44 can also be able to selectively grant or deny access to a specified area in the memory 32 to a specified PCIe device 36.

[0028] In some implementations, memory fetch device 24 includes interface circuitry 52 for communicating with, for example, RC 40 via a PCIe bus, and memory fetch circuitry 56 for performing the methods described herein.

[0029] like Figure 1 The configuration of system 20 and device 24 shown in is an example configuration shown purely for conceptual clarity. Any other suitable configuration may be used in alternative embodiments. For example, the memory acquisition device may be connected to the PICe bus via a PCIe switch.

[0030] The embodiments described herein are primarily focused on the task of performing secure and trusted memory access, i.e., reading data from selected areas of memory 32 by device 24. The manner in which the data is analyzed for the purpose of detecting malware or for any other purpose, and the responsive actions that may be taken if tampering with the memory access is detected, are considered outside the scope of the present disclosure.

[0031] In some embodiments, the analysis and / or response actions are performed by the device 24 itself. In other embodiments, the device 24 outputs the read data to an external system for further processing (e.g., analysis and / or response). The external system may include, for example, a system for forensic investigation, a system information and event management (SIEM) system, a virtual machine (VM) honeypot, a malware analysis system, an antivirus system, a rootkit detection system, a firewall, an emergency response system, a performance management system, a patch management system, an update management system, an intrusion detection system, an intrusion prevention system, or any other suitable system.

[0032] The various elements of the system 20 and device 24 may be implemented using suitable hardware, such as one or more application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs), using software, or using a combination of hardware and software elements. In some embodiments, the CPU 28 and circuitry 56 may include one or more general purpose processors that are programmed by software to perform the functions described herein. The software may be downloaded to any processor in electronic form, such as over a network, or it may be provided and / or stored alternatively or additionally on a non-transitory tangible medium such as a magnetic, optical, or electronic memory.

[0033] Trusted tamper-resistant memory acquisition by PIC-E devices

[0034] As described above, the fact that table 48 is controlled by software (e.g., an operating system or hypervisor) that may be compromised creates a security risk that may be exploited, for example, by malware. In some embodiments, circuitry 56 of memory acquisition device 24 overcomes this risk by bypassing the address translation stage entirely. Such a bypass is possible because, in order to perform a memory acquisition, device 24 only needs to know the physical memory address in memory 32. Device 24 is typically unaware of the bus address space.

[0035] Figure 2 1 is a flow chart schematically illustrating a method for tamper-resistant memory acquisition according to an embodiment of the present invention. Conventional read / write requests and read requests issued by the disclosed memory acquisition device are shown together in the figure. The left hand side of the flow chart (left side of the vertical dashed line) is performed by the memory acquisition device 24. The right hand side of the flow chart (right side of the vertical dashed line) is depicted for reference.

[0036] The method begins with each PCIe device issuing a read request to read data from the memory 32 and / or a write request to write data to the memory 32 in a request step 60. In particular, the memory acquisition circuit 56 of the device 24 issues a read request as described above. Each read request issued by the device 24 specifies a specific physical memory address of the memory 32 to be read. The address type (AT) field of each read request is set to "10b", indicating that the address in the read request is the result of an address translation (although no address translation transaction of a physical memory address is generated prior to the read request).

[0037] Referring to the right hand side of the figure: if the AT field of the read request is not '10b', as checked in the type check step 64, two cases are possible, as depicted in the pass-through check step 68. If the pass-through (PT) value of the request is '0', then the memory is accessed according to the bus address in the bus address access step 72. If the PT value of the request is '1', then the memory is accessed according to the physical memory address in the physical address access step 76. In both cases (i.e., whenever the AT is different from '10b'), the requesting device cannot guarantee that the request is actually serviced from the requested physical memory address.

[0038] Referring to the left hand side of the drawing, this is performed when the AT field of the read request is '10b' according to an embodiment of the present invention. In a denial check step 80, the memory acquisition circuit 56 of the device 24 checks whether any read request has been denied or rejected by the IOMMU. If so, in a response step 84, the circuit 56 treats the denial as a suspected tampering operation and initiates a response action. The circuit 56 can perform or trigger any suitable response action, such as, for example, issuing an alert to an operator (and / or to the system), or shutting down or isolating the entire computing system or some part thereof.

[0039] If the IOMMU does not deny any read request, then circuitry 56 concludes that no tampering has occurred. Therefore, in a sending step 88, circuitry 56 may send the data retrieved from memory 32 in response to the read request for further analysis. Any suitable form of analysis may then be applied.

[0040] Although the embodiments described herein are primarily directed to PCIe and ATS, the methods and systems described herein may also be used with other suitable bus protocols such as Universal Serial Bus (USB), Thunderbolt, or any other suitable protocol. Although the embodiments described herein are primarily directed to memory acquisition for the purpose of memory analysis, the disclosed techniques may also be used for any other suitable application.

[0041] Therefore, it should be understood that the above embodiments are cited by way of example, and the present invention is not limited to the above specific illustrations and descriptions. On the contrary, the scope of the present invention includes the combination and sub-combination of the above-mentioned various features, as well as changes and modifications to the features that are thought of by those skilled in the art when reading the foregoing description and are not disclosed in the prior art. The documents incorporated by reference into this patent application should be considered as part of this application, unless the definition of any item in these incorporated documents conflicts with the definition explicitly or implicitly made in this specification, in which case only the definition in this application should be considered.

Claims

1. A device for memory acquisition in a computer system based on an input-output memory management unit, comprising: an interface configured to communicate over a bus operating according to a bus protocol, wherein the bus protocol supports address translation transactions that translate between bus addresses in an address space of the bus and physical memory addresses in an address space of a memory; as well as a memory acquisition circuit configured to read data from the memory by issuing one or more requests on the bus using the bus protocol, the one or more requests (i) specifying an address to be read in terms of the physical memory address, and (ii) indicating that the physical memory address in the request has been translated from a corresponding bus address, although the address has not been obtained through any address translation transaction on the bus, wherein the memory acquisition circuit is configured to detect malicious operation by detecting that one or more of the requests have been denied and to perform a responsive action in response to detecting that one or more of the requests have been denied, The response action includes at least one of: issuing an alarm to an operator of the device; and shutting down or isolating a portion or all of the device.

2. The device according to claim 1, wherein: By indicating that the physical memory address in the request has already been translated, the memory fetch circuitry is configured to prevent performance of a software controlled address translation as part of servicing the request.

3. The device according to claim 1, wherein: By indicating that the physical memory address in the request has been translated, the memory fetch circuitry is configured to ensure that the request will be serviced entirely in hardware.

4. A computer system for memory acquisition in a computer system based on an input-output memory management unit, comprising: a memory accessible via a bus interface according to a bus protocol; an input-output memory management unit configured to connect a peripheral device to the memory via the bus interface, including performing an address translation transaction for translating between a bus address in an address space of the bus interface and a physical memory address in an address space of the memory; and a trusted peripheral device configured to read data from the memory by issuing one or more requests on the bus interface using the bus protocol, the one or more requests (i) specifying an address to be read in terms of the physical memory address, and (ii) indicating that the physical memory address in the request has been translated from a corresponding bus address, although the address has not been obtained through any address translation transaction on the bus, wherein the trusted peripheral device is configured to detect malicious operation by detecting that one or more of the requests have been denied by the input-output memory management unit and to perform a response action in response to detecting that one or more of the requests have been denied, The response action includes at least one of the following: issuing an alarm to an operator; and shutting down or isolating a part or all of the computer system.

5. The computer system according to claim 4, wherein: By indicating that the physical memory address in the request has been translated, the peripheral device is configured to prevent the input-output memory management unit from performing a software-controlled address translation as part of servicing the request.

6. The computer system according to claim 4, wherein: By indicating that the physical memory address in the request has been translated, the peripheral device is configured to ensure that the input-output memory management unit services the request entirely in hardware.

7. A method for memory acquisition in a computer system based on an input-output memory management unit, comprising: reading data from a memory by issuing one or more requests on a bus, the bus operating according to a bus protocol supporting address translation transactions for translating between bus addresses in an address space of the bus and physical memory addresses in an address space of the memory, wherein the request (i) specifies an address to be read in terms of the physical memory address, and (ii) indicates that the physical memory address in the request has been translated from a corresponding bus address, although the address has not been obtained by any address translation transaction on the bus; as well as initiating a responsive action in response to detecting that one or more of the requests have been denied, The response action includes at least one of the following: issuing an alarm to an operator; and shutting down or isolating a part or all of the computer system.

8. The method of claim 7, wherein issuing the request comprises preventing software-controlled address translation from being performed as part of servicing the request.

9. The method of claim 7, wherein issuing the request comprises ensuring that the request will be serviced entirely in hardware.

Images