Input / output memory management unit

Abstract

An input / output memory management unit (IOMMU) comprises address translation circuitry to control address translation in response to input / output memory system transactions issued by one or more input / output devices; at least one dirty page tracking base address register to store a base address of a dirty page buffer structure for tracking addresses of dirty pages of address space; and dirty page tracking circuitry to: detect whether an input / output memory system transaction associated with a given translation context for which dirty page tracking is enabled causes a transition of a given page of address space from clean to dirty, and in response to detecting the input / output memory system transaction associated with the given translation context for which dirty page tracking is enabled causing a transition of the given page from clean to dirty, issue a write memory access request requesting that a location within a selected dirty page buffer structure, identified using the base address indicated in a selected dirty page tracking base address register, is updated to identify a page address of the given page.

Description

[0001] INPUT / OUTPUT MEMORY MANAGEMENT UNIT

[0002] The present technique relates to the field of input / output memory management units (lOMMUs).

[0003] A data processing apparatus may have a memory management unit (MMU) for managing accesses to memory. For example the MMU may be responsible for translating virtual addresses specified by a device wishing to access memory into physical addresses which directly identify the locations to access in memory. The MMU may also control whether a device is allowed to access the requested address based on access permissions set for regions of the address space. A processing element capable of instruction execution, such as a central processing unit, may have its own MMU for managing access to memory in response to memory access transactions issued by the processing element. However, a processing system may also have input / output (I / O) devices which are provided with direct memory access to the memory system shared with one or more processing elements. Such I / O devices do not typically have their own MMU. Nevertheless, supporting address translation and other memory management functions, such as access permissions checking, for input / output memory access transactions (e.g. read / write transactions) issued by I / O devices can be helpful to avoid exposing physical memory directly to devices. This can be beneficial both for supporting virtual memory (allowing fragmentation of the memory used by a device across discontiguous physical memory regions), and for security reasons (allowing access permissions checks to be imposed so that malicious devices cannot compromise data in memory not allocated for the device). Hence, it can be useful to provide an input / output memory management unit (IOMMU), also referred to as a system memory management unit (SMMU), which provides memory management functions including address translation for I / O memory access transactions originating from I / O devices.

[0004] At least some examples of the present technique provide an input / output memory management unit (IOMMU), the IOMMU comprising: address translation circuitry to control address translation in response to input / output memory system transactions issued by one or more input / output devices; at least one dirty page tracking base address register to store a base address of a dirty page buffer structure for tracking addresses of dirty pages of address space; and dirty page tracking circuitry to: detect whether an input / output memory system transaction associated with a given translation context for which dirty page tracking is enabled causes a transition of a given page of address space from clean to dirty, and in response to detecting the input / output memory system transaction associated with the given translation context for which dirty page tracking is enabled causing a transition of the given page from clean to dirty, issue a write memory access request requesting that a location within a selected dirty page buffer structure, identified using the base address indicated in a selected dirty page tracking base address register, is updated to identify a page address of the given page. At least some examples of the present technique provide computer-readable code for fabrication of an IOMMU as described above. The computer-readable code may be stored on a storage medium. The storage medium may be a non-transitory storage medium.

[0005] At least some examples of the present technique provide a method comprising: controlling address translation in response to input / output memory system transactions issued by one or more input / output devices; detecting whether an input / output memory system transaction associated with a given translation context for which dirty page tracking is enabled causes a transition of a given page of address space from clean to dirty, and in response to detecting the input / output memory system transaction associated with the given translation context for which dirty page tracking is enabled causing a transition of the given page from clean to dirty, issuing a write memory access request requesting that a location within a selected dirty page buffer structure, identified using a base address indicated in a selected dirty page tracking base address register, is updated to identify a page address of the given page.

[0006] At least some examples provide a computer program comprising instructions which, when executed on a host data processing apparatus, control the host data processing apparatus to simulate an input / output memory management unit (IOMMU), the computer program comprising: address translation program logic to control the host data processing apparatus to perform address translation in response to simulated input / output memory system transactions issued by one or more simulated input / output devices; and dirty page tracking program logic to: detect whether a simulated input / output memory system transaction associated with a given translation context for which dirty page tracking is enabled causes a transition of a given page of simulated address space from clean to dirty, and in response to detecting the simulated input / output memory system transaction associated with the given translation context for which dirty page tracking is enabled causing a transition of the given page from clean to dirty, issue a write memory access request requesting that a location within a selected dirty page buffer structure, identified using a base address indicated in a selected simulated dirty page tracking base address register, is updated to identify a page address of the given page. The computer program may be stored on a storage medium. The storage medium may be a non-transitory storage medium.

[0007] Further aspects, features and advantages of the present technique will be apparent from the following description of examples, which is to be read in conjunction with the accompanying drawings, in which:

[0008] Figure 1 schematically illustrates an example of a data processing system in which a number of input / output devices access memory through an input / output memory management unit (IOMMU);

[0009] Figure 2 schematically illustrates an example where the IOMMU is distributed across several devices;

[0010] Figure 3 illustrates an example of an IOMMU (SMMU) comprising dirty page tracking circuitry; Figure 4 illustrates steps for dirty page tracking;

[0011] Figure 5 illustrates an example of some registers of the IOMMU;

[0012] Figure 6 illustrates an example of control fields used by the IOMMU to control dirty page tracking;

[0013] Figure 7 illustrates use of two dirty page buffer structures to track dirty pages;

[0014] Figure 8 illustrates steps for hardware-triggered and software-triggered switching of which dirty page buffer structure is active; and

[0015] Figure 9 illustrates a simulation example.

[0016] An input / output memory management unit (IOMMU) comprises address translation circuitry to control address translation in response to input / output memory system transactions issued by one or more input / output devices. The IOMMU has at least one dirty page tracking base address register to store a base address of a dirty page buffer structure for tracking addresses of dirty pages of address space. The IOMMU also has dirty page tracking circuitry which detects whether an input / output memory system request associated with a given translation context for which dirty page tracking is enabled causes a transition of a given page of address space from clean to dirty. In response to detecting the input / output memory system request associated with the given translation context for which dirty page tracking is enabled causing a transition of the given page from clean to dirty, the dirty page tracking circuitry issues a write memory access request requesting that a location within a selected dirty page buffer structure, identified using the base address indicated in a selected dirty page tracking base address register, is updated to identify a page address of the given page.

[0017] This can be helpful for accelerating live migration of a virtual machine from a source host machine to a destination host machine. Dirty page tracking can be useful to reduce the downtime associated with migrating a virtual machine, to allow detection of which pages have been updated during a phase of copying pages of data to the destination while the virtual machine remains running. By providing the IOMMU with a register which specifies the base address of a dirty page buffer structure stored in memory, and dirty page tracking circuitry (implemented in hardware) which records, to the dirty page buffer structure, an indication of page addresses of pages which become dirty as a result of I / O memory access requests, this can provide hypervisor software (the software that is responsible for live migration of a virtual machine) with a log of dirty pages which enables more efficient identification of the newly dirty pages which might require further copying in a subsequent round of a pre-copy phase of the live migration process. This can help reduce the overhead of live migration in a system comprising an IOMMU.

[0018] The IOMMU may comprise at least one producer index register each to store a producer index value indicative of an entry of a corresponding dirty page buffer structure to be updated next. The dirty page tracking circuitry may identify the location within the selected dirty page buffer structure (that is to be updated to identify the page address of the given page) using the base address indicated in the selected dirty page tracking base address register and the producer index value indicated in a corresponding producer index register corresponding to the selected dirty page tracking base address register. This helps the dirty page tracking circuitry maintain a compact list of page addresses. The producer index value may be incremented by the dirty page tracking circuitry (under hardware control, without needing explicit software intervention for each increment) each time a new entry identifying a page address is written to the dirty page buffer structure.

[0019] In some examples, the producer index register and the dirty page tracking base address register relating to a given dirty page buffer structure could be the same register, if there are sufficient bits available within the dirty page tracking base address register to specify the producer index value. However, other examples may provide separate registers to specify the base address and producer index value respectively.

[0020] In some examples, an address of the location to be updated in the selected dirty page buffer structure may be determined to have a value equivalent to the sum of the base address and an offset value dependent on the producer index value. This does not necessarily need an addition to be performed - if the base address is naturally aligned to a memory address boundary, calculation of the address of the location to be updated could be performed by concatenating the base address and offset value. The offset value could be a multiple of the producer index value itself or a multiple of an adjusted version of the producer index value (e.g. an incremented version of the producer index value), depending on the particular scheme used to track the producer index value (e.g. depending on whether the producer index value is updated before or after the generation of the write request for updating the dirty page buffer structure). The multiple used to form the offset value may correspond to the size of one entry of the dirty page buffer structure.

[0021] In some examples, the IOMMU comprises a plurality of dirty page tracking base address registers to store base addresses of a plurality of dirty page buffer structures. The dirty page tracking circuitry may select, as the selected dirty page buffer structure, an active dirty page buffer structure indicated by dirty page buffer status information as being currently in use by the dirty page tracking circuitry.

[0022] IOMMU support for multiple alternative dirty page buffer structures (with selection of which of the buffers is currently active for being updated in hardware by the dirty page tracking circuitry) can be extremely helpful for accelerating live migration of a virtual machine in a system comprising I / O devices with direct memory access to the shared memory used by the virtual machine. When a buffer of page addresses of dirty pages is maintained in memory by the dirty page tracking circuitry, then after a period of tracking the dirty pages, eventually the buffer is to be handed over to software for processing (e.g. for the software to identify which pages will require further copying in a subsequent round of copying). At the point when the software is processing the contents of the buffer, it is undesirable for the dirty page tracking circuitry to continue making updates to the buffer. If the dirty page tracking was implemented at a processing element capable of instruction execution, it would be relatively simple to ensure that a single buffer is not updated in the period when it is being consumed by software, simply by sending an interrupt to the processing element to temporarily halt processing of the migrated virtual machine for a short period while software parses the buffer. However, a problem specific to lOMMlls is that it is not possible to interrupt I / O devices in a similar manner to processing elements, and so during a live migration process a dirty page buffer should remain available for being updated to track pages dirtied by I / O memory system transactions, even while software is parsing information from a previously populated dirty page buffer. Hence, to enable dirty page tracking when supporting I / O use cases, it is helpful to support at least two distinct dirty page buffer structures associated with corresponding base address registers, with a given one of those structures being designated as active at a given time. The active structure is updated by the dirty page tracking hardware of the IOMMU, while other structures can be handed over to software for processing while I / O driven dirty page tracking is still ongoing using the active structure.

[0023] The dirty page tracking circuitry may be responsive to occurrence of a dirty page buffer switch event, to switch which of at least two dirty page buffer structures is the active dirty page buffer structure. Hence, from time to time, the dirty page tracking circuitry switches which buffer is active, e.g. to allow the previously active buffer to be processed by software while updates of dirty page addresses continue in the newly active buffer.

[0024] The IOMMU may support one or more distinct types of dirty page buffer switch event.

[0025] In some examples, the dirty page tracking circuitry may support a software-triggered dirty page buffer switch event triggered by a software request to switch which of the at least two dirty page buffer structures is the active dirty page buffer structure. Even if the currently active dirty page buffer structure is not full yet, to support live migration use cases it can be helpful to provide architectural support for software to be able to trigger a switch of which buffer is active at an arbitrary software-determined timing (rather than relying purely on hardware decision of which buffer is currently active). For example, once software has finished processing the contents of a previous buffer, it may help reduce the overall live migration process latency if the software can immediately switch which buffer is active and start processing the contents of the buffer which was active before the switch, without waiting for an alternative hardware-triggered switch event to occur (such as the currently active buffer becoming full).

[0026] The software request can be implemented in different ways. In general, the IOMMU may have some control state information which can be updated by software to cause the software request to be detected. This control state information can be represented in a variety of formats. However, in some examples, the software request comprises a request to update dirty page buffer validity information associated with a given dirty page buffer structure which is currently the active dirty page buffer structure, to indicate that the given dirty page buffer structure should become invalid. The switching of a valid buffer structure to invalid may be detected as the trigger for the hardware to start switching to another buffer becoming active. However, the software request may not necessarily trigger the dirty page tracking circuitry to switch immediately to directing its dirty page information updates to a new active buffer. There may still be some outstanding requests in progress, e.g. write requests which are not yet complete and may still update the buffer that was previously active before the software requested the switch of active buffer. Therefore, in some examples, in response to the software request, the dirty page tracking circuitry may (i) complete any outstanding updates to the given dirty page buffer structure; (ii) in response to completion of any outstanding updates to the given dirty page buffer structure, update a software-accessible acknowledgement indication; and (iii) switch which of said at least two dirty page buffer structures is selected as the active dirty page buffer structure. The acknowledgement indication can be helpful to signal to software when it is safe to start processing the outgoing dirty page buffer structure (to reduce risk that outstanding updates to the outgoing active buffer structure are lost because software started processing the buffer before those outstanding updates were completed).

[0027] The dirty page tracking circuitry may support a hardware-triggered dirty page buffer switch event. For example, the hardware-triggered dirty page buffer switch event may comprise detection that the active dirty page buffer structure has become full. Hence, when the current active buffer structure becomes full, the hardware can automatically switch to an alternative buffer structure (identified using another of the dirty page tracking base address registers) to enable updates to continue. In some examples, as well as switching to the alternative buffer, when the active buffer becomes full, the dirty page tracking circuitry may also signal an interrupt to at least one processing element, to provide a notification to software that the previously active buffer is ready to process.

[0028] In some examples, the dirty page tracking circuitry may detect whether the active dirty page buffer structure has become full based on: a producer index value indicative of a next entry to update in the active dirty page buffer structure; and a software-programmable size parameter indicating a maximum size of the active dirty page buffer structure. Hence, the IOMMU may support variable size buffers, which can be helpful to allow more write-intensive workloads (workloads with high memory throughput) to be migrated.

[0029] In some examples, the dirty page tracking circuitry may support both: a software-triggered dirty page buffer switch event triggered by a software request to switch which of the at least two dirty page buffer structures is the active dirty page buffer structure; and a hardware-triggered dirty page buffer switch event. Supporting both types of dirty page buffer switch event helps better support live migration use cases as it provides a hardware mitigation against buffer overflow while also allowing software to signal to the dirty page tracking hardware when it wishes a buffer switch to occur so that software can process the previously active buffer.

[0030] In some examples, the dirty page buffer structure has a configurable size defined by a software-programmable size parameter. For example, the software-programmable size parameter may be defined by a control value in a control register of the IOMMU. The control register may be accessible to software executing on a processing element of the processing system comprising the IOMMU, e.g. by the control register being memory-mapped so that software can access it by issuing a store request to store data to an address mapped to the control register.

[0031] In some examples, the dirty page tracking circuitry is configured to determine whether dirty page tracking is enabled for the given translation context based on a stream table entry defining translation control information for the given translation context. The IOMMU may have a stream table base address register identifying a base address of a stream table structure stored in memory, the stream table structure comprising a number of stream table entries each providing translation control information for a respective translation context. I / O memory system transactions issued by I / O devices to the IOMMU may specify a stream identifier of the corresponding translation context, to allow the IOMMU to select the associated translation control information from the corresponding stream table entry for that stream identifier. For example, a stream table entry may specify control information such as a base address of a translation table structure used to provide address mappings for translating addresses in the corresponding translation context, and translation table format information which controls how page table walks of the translation tables are performed. Hence, as the stream table entry records information specific to a given translation context, it can be efficient to also use the stream entry to specify a value indicating whether dirty page tracking is enabled or disabled for the given translation context. This enables the overhead of dirty page tracking to be reserved for those translation contexts for which dirty page tracking is needed, rather than being performed for all translation contexts. For example, dirty page tracking may be enabled for a context associated with a virtual machine undergoing live migration. The stream table entry may be programmable by software to select whether dirty page tracking is enabled.

[0032] As mentioned above, for IOMMU use cases it can be helpful to provide base address registers and producer index registers to support at least two buffers which can alternately be selected as an active dirty page buffer structure to enable software handover of one buffer while hardware updates continue in another buffer. Although it is possible to provide three or more such buffers which are treated as a group for which only one buffer is active at a given time, this may be unnecessary and a pair of two, alternately active, buffers may be sufficient to enable continued dirty page tracking in response to I / O transactions while software is processing another buffer.

[0033] In some examples, it is possible to provide multiple pairs (or groups) of such buffers, each pair (or group) being allocated for tracking pages which become dirty in response to I / O transactions associated with a respective translation context. For example, multiple sets of base registers and producer index registers may be provided (each set comprising at least two base registers and at least two producer index registers to define the base address and producer index value for a corresponding pair or group of buffers). This can help support multiple live migration processes running in parallel for migrating two different virtual machines. The stream table entry for a given translation context could specify which of the respective pairs or groups of buffers is to be used for tracking dirty page addresses for that context. If multiple independent pairs or groups of buffers are supported, then the dirty page tracking circuitry may select an active buffer per pair or group (e.g. one buffer selected from a first pair of buffers may be active for tracking dirty pages for I / O requests associated with a first translation context, while another buffer selected from a second pair of buffers may be active for tracking dirty pages for I / O requests associated with a second translation context).

[0034] Also, in some examples the IOMMU may comprise a plurality of dirty page tracking base address registers to store base addresses of respective dirty page buffer structures corresponding to different security states. The dirty page tracking circuitry may select the selected dirty page buffer structure based on an associated security state associated with the input / output memory system transaction. This can be helpful for confidential computing use cases where it may be desirable to ensure that software executing in one security state cannot learn information about the activity of software executing in another security state. For implementations supporting multiple security states, there may be a respective dirty page buffer structure active for each security state (e.g. one buffer may be active for tracking dirty pages for I / O requests associated with a first security state, while another buffer may be active for tracking dirty pages for I / O requests associated with a second security state).

[0035] The dirty pages could be tracked in different ways. In some examples, the dirty pages could be tracked based on virtual page addresses.

[0036] However, the address translation circuitry may support two-stage address translation in which translation of a virtual address to a physical address depends on first-stage address translation information mapping the virtual address in a virtual address space to an intermediate address in an intermediate address space and second-stage address translation information mapping the intermediate address in the intermediate address space to the physical address in a physical address space; and the dirty page tracking circuitry may maintain the dirty page buffer structure to identify the dirty pages of address space using intermediate page addresses of the intermediate address space. Tracking the dirty pages based on intermediate page addresses can be helpful for supporting live migration use cases, because it reduces the need for the hypervisor to consider different first-stage translation tables in use within the same virtual machine when copying pages from the source machine to the destination machine, making the live migration control more efficient from the hypervisor’s point of view.

[0037] In some examples, the at least one dirty page tracking base address register is separate from a base address structure (e.g. stream table) for identifying a base address of a translation table structure used to provide address translation mapping information for the address translation controlled by the address translation circuitry. In some examples, the dirty page tracking circuitry maintains, as the dirty page buffer structure, a list of page addresses of the dirty pages of address space. For example, the list may be managed as a FIFO (first in first out) buffer or queue. Each entry of the list may specify a multi-bit numeric value corresponding to the (intermediate) page address of a corresponding dirty page of address space. This provides a more compact and easy to parse representation of the dirty page addresses to be obtained than would be possible using dirty page flags stored in the corresponding translation table entries of the translation table structure.

[0038] The techniques discussed above may be implemented within an apparatus which has hardware circuitry provided for implementing the IOMMU discussed above. However, the same technique can also be implemented within a computer program which executes on a host data processing apparatus to control the host to simulate the IOMMU (either as a standalone component or as part of a simulation of a wider processing system). Such a computer program may control the host data processing apparatus to simulate the hardware of the IOMMU, which can be helpful for testing, design or performance analysis of the IOMMU hardware, and to enable software which is to interact with the IOMMU to be tested before any real IOMMU hardware is available yet. Hence, the computer program may have address translation program logic and dirty page tracking program logic which simulates the behaviour of the address translation circuitry and dirty page tracking circuitry described earlier. The simulation program may support corresponding features to those described for the IOMMU above.

[0039] Specific examples are now set out with reference to the drawings.

[0040] Figure 1 schematically illustrates an example of an apparatus (e.g. a data processing system, integrated circuit or system on chip, or collection of interconnected chiplets) having at least one processor 20 (in this example, three CPUs (Central Processing Units), but other examples of processors include a GPU (Graphics Processing Unit) or NPU (Neural Processing Unit - a type of processor with specialized hardware for accelerating vector and / or matrix operations or other operations used in Neural Network and other machine learning processing). Each processor may have an internal memory management unit (MMU) which functions as address translation circuitry for translating virtual addresses specified by instructions executed by the processor into physical addresses identifying locations within the memory system. The MMU may have at least one TLB (translation lookaside buffer) for caching translation information which depends on translation table data from translation table structures (also known as page table structures) stored in the memory system. The page table structures define the address mappings between virtual and physical addresses and may also define memory access permissions which may define whether certain software processes executing on the processor are allowed to access certain addresses.

[0041] In addition to the processor or other devices capable of instruction execution which have their own internal MMU, the system may also include one or more input / output (I / O) devices 40, 50 which may not have an internal MMU, and so for accessing memory and providing address translation functionality, such devices may communicate with the rest of the system via an IOMMU 60, also referred to hereafter as a system memory management unit (SMMLI), which includes address translation circuitry which controls address translation and memory permissions based on translation data defined in page table structures in memory (the page table structures being programmable by software executing on the CPUs 20). Again, the SMMU 60 may have one or more TLBs 62 which have a similar functionality to the TLB within the MMU of the processor 20, for caching information derived from the page table structures. The I / O devices 40, 50 which access memory via the SMMU 60 can include cached devices which include an internal cache and uncached devices which do not have any cache. For example, a device 40, 50 could include a display controller for controlling display of image frames on display, a network controller for controlling input or output of data via a network, a storage controller for transferring data to or from external data storage, and many other kinds of I / O device.

[0042] All of the units 20, 40, 50 communicate with each other via an interconnect 10 which is responsible for routing transactions between the requester devices and memory 30. Interconnect 10 may also be responsible for managing coherency between data cached in respective caches of the system, which may include private caches (private to a particular memory system requester) within the CPUs 20 or devices 40, 50 and shared caches 32 (shared between multiple requesters) which are associated with the interconnect 10 or other parts of the memory system. It will be appreciated that Figure 1 is a simplified diagram and the system may have many other components not shown in Figure 1 for conciseness.

[0043] The TLB 62 of the SMMU 60 caches recently or commonly used translations between virtual and physical memory addresses. So, as a first step in an address translation process, the TLB 62 is consulted to detect whether the TLB already contains the required address translation. If not, then a more involved translation process may be used, for example involving consulting so-called page tables (translation tables) holding address translation information, typically resulting in the TLB 62 then being populated with details of the required translation.

[0044] The virtualized system of Figure 1 may make use of multiple stage address translation. A virtual address (VA) required by an executing program or other system module in one of the devices 40 is translated to an intermediate physical address (IPA) by a first MMU stage (under control of first-stage address translation mappings defined by a first set of page tables). As far as the VM software is aware, the IPA is a physical address used to access the system memory. However, the virtualized system provides a second level of address translation such that the IPA is then translated to a physical address (PA) by a second MMU stage (under control of second- stage address translation mappings defined by a second set of page tables). The IPA to PA translation is wholly under the control of the hypervisor. So, a particular VM may be exposed to the VA to IPA translation, whereas the hypervisor has oversight of the IPA to PA translations and therefore control of real accesses to the physical system memory. The TLBs 62 may implement separate stage-1 and stage-2 TLB structures caching the first and second stages of translation respectively, or could be implemented as a combined stage-1 -and-2 TLB which caches a mapping from a virtual address direct to a physical address (that mapping depending on both the first and second sets of page tables associated with the first and second MMU stages).

[0045] Figure 1 shows the use of one SMMLI 60. In other examples, such as that shown in Figure 2, the SMMLI (IOMMU) functionality may be, in part, distributed in that TLBs may be respectively associated with each device 40, 50 and may be coupled to each other and a main MMU via either the memory system interconnect 10 or a dedicated MMU bus 64 (in the case of an internal MMU bus 64, at least the central MMU portion of the distributed SMMU 60 is coupled to the system interconnect 10 to enable access to the underlying memory 30).

[0046] In the example shown in Figure 2, certain translations which are able to use translation information cached locally in a TLB of that device (which may be referred to as a micro TLB (pTLB)) may take place locally to the device. However, even if certain translations may take place locally at certain devices, a central address translation component of the SMMU 60 is still provided to perform page table walks for obtaining the translation information used to populate the pTLBs. If a required translation is not found in the respective pTLB (a TLB “miss”), then an address translation request is issued to the central address translation circuitry specifying a first address and requesting translation of the first address. The central address translation circuitry translates the first address into a second address using page table information specified in a TLB or in memory, in the same way as the SMMU shown in Figure 1. The SMMU of Figure 2 may therefore be configured in the same way as the SMMU of Figure 1. If the central address translation circuitry performs the address translation without encountering a page fault, then it returns an address translation response to the device, the address translation response comprising the second address and enabling the device to cache the translation from the first address to the second address locally in its pTLB.

[0047] The SMMU 60 is provided with architectural features which support acceleration of live migration of a virtual machine. In data centre use cases, for example, it can be useful to support live migration where a virtual machine is migrated from a source machine to a destination machine. Live migration is a software-driven technique in which, to reduce the performance degradation and downtime during which the virtual machine is out of action during the migration, the live migration process typically includes a pre-copy phase and a stop-and-copy phase and the pre-copy phase is performed while the virtual machine is still running. In the pre-copy phase, multiple rounds of copying are typically performed. Each round comprises copying data or program code of memory regions associated with the virtual machine to memory associated with the destination machine, while tracking pages which become dirtied during the copying. Each subsequent round of the pre-copy phase comprises copying the pages which were detected as becoming dirty in the previous round. It is expected that each successive round of pre-copying is likely to require fewer pages to be copied than the previous round, so that over several rounds the number of pages to be copied per round reduces and reduces until eventually, once the number of dirty pages detected in a given pre-copy round is less than a given threshold, the virtual machine is halted and the stop-and-copy phase copies the remaining dirty pages from source to destination. Hence, the pre-copy phase helps to reduce the downtime to within acceptable limits, by ensuring there is a maximum bound on the number of pages to be copied in the stop-and-copy phase while the virtual machine is halted.

[0048] However, the pre-copy phase relies on tracking the pages which transition from clean to dirty while copying is being performed in the background of continued functioning of the virtual machine. Pages used by the virtual machine could become dirty due to writes triggered by I / O devices, not just writes triggered by the processing element executing the virtual machine. A typical approach for dirty page tracking by an IOMMU is to set write access flags within the page table entries used to provide address translation mappings for the dirtied pages. However, such page table entries are typically recorded in a non-compact manner within multi-level page table structures implemented using a tree structure, for which locating the entry corresponding to a given page requires traversal of multiple levels of page tables by following a trail of pointers. Therefore, it can be extremely inefficient for the hypervisor software managing the live migration process to use the page tables to identify the pages which have become dirty, as this would require scanning of all branches of the page table structure associated with the migrated virtual machine to find the page table entries for each page accessible to that virtual machine and identify which pages are dirty. This can greatly prolong the live migration process, harming system performance and causing a delay in being able to complete the live migration process (which may therefore also delay other processes which may be waiting for the live migration process to complete so that they can use resource associated with the source machine 20 from which the virtual machine was migrated).

[0049] Figure 3 illustrates an example of some features of the SMMLI 60 for hardware acceleration of dirty page tracking, which can help support more efficient live migration as one example use case. The SMMLI 60 includes address translation circuitry 70 for controlling address translation in response to I / O memory system transactions received from I / O devices 40, 50. SMMLI 60 includes a number of control registers 80, which includes at least one dirty page tracking base address register 86 providing a base address of a data structure in memory designated as a dirty page buffer structure for tracking addresses of dirty pages of memory address space. The control registers 80 also include at least one producer index register 86 (each producer index register corresponding to a respective one of the base address registers 86), for providing a producer index value for tracking the next entry of the dirty page buffer structure to be updated with an address of a dirty page. The SMMLI 60 also has dirty page tracking circuitry 90, implemented using hardware circuit logic, which is responsive to detection of a clean-to-dirty transition of a given page of address space, to cause an intermediate address of the newly dirty page to be written to a memory system location corresponding to an address derived from the base address and producer index (obtained from registers 84, 86) associated with a currently active dirty page buffer structure. By implementing the dirty page buffer as a contiguous list of addresses of dirty pages, rather than as a fragmented set of dirty bits distributed across individual page table entries at the leaves of the tree structure of the page tables used for address translation, this enables a denser log of dirty pages to be obtained which is much more efficient for a hypervisor to parse during live migration processing.

[0050] It is noted that live migration is a software-driven process, so a hardware system comprising the SMMLI 60 described above provides the architectural functionality to enable software executing on the hardware system, if desired by the software developer, to perform live migration with improved performance compared to an apparatus that does not support the SMMLI having the dirty page tracking functionality 84, 86, 90. However, the same hardware platform can also be used to execute with software which does not require live migration to be performed. Hence, the live migration process itself is not an essential feature of the hardware platform. A hardware platform comprising the SMMLI 60 described above is an improved platform compared to a platform not supporting the dirty page tacking circuitry 90 and associated control register state, as it gives software developers the option to perform live migration in a more performanceefficient manner which would not otherwise be possible. Whether software developers choose to make use of that architectural functionality is a design choice for the software developers and so is not an essential feature of the architecture of the hardware platform. Similarly, although the dirty page tracking described above can be particularly useful for supporting live migration use cases, some software developers may find other purposes for which it is useful to support dirty page tracking, and so the dirty page tracking circuitry can similarly accelerate those other use cases compared to an approach relying on tracking dirty pages using the page tables.

[0051] Figure 4 illustrates steps for dirty page tracking. At step 100, an I / O memory system request (transaction) is detected that causes a transition of a given page of address space from clean to dirty. For example, the transition from clean to dirty may be detected when a write I / O request is received that targets a page of address space for which a dirty flag (or other kind of write access tracking flag) within the page table entry that provides the address translation mapping for that page is currently clear, indicating that there has been no previous write to the same page since the dirty I write access tracking flag was cleared in the page table entry. The clean-to-dirty transition may cause the SMMLI 60 to set the dirty / write access tracking flag in the corresponding page table entry, so that subsequent writes will not cause a clean-to-dirty transition to be detected.

[0052] At step 102, the dirty page tracking circuitry 90 determines whether dirty page tracking is currently enabled for the translation context associated with the received I / O memory system request. If not, then no update is required to any dirty page tracking buffer structure.

[0053] If dirty page tracking is determined to be enabled for the translation context associated with the I / O memory system request detected as causing the clean-to-dirty transition, then at step 104 the dirty page tracking circuitry 90 issues a memory access request requesting that a location within a selected dirty page buffer structure is updated to identify a page address of a given page of address space targeted by the I / O memory system request. For example, the address of the location at which the buffer is updated is determined as a function of the base address and producer index value defined in registers 84, 86 for a currently active dirty page buffer.

[0054] In one example, the address of the location to be updated may be determined as logically corresponding to B + M * (P + x), where B is the base address, P is the producer index value, M is a multiple corresponding to a size of one entry of the dirty page buffer, and x is an adjustment value to account for whether the absolute value of the producer index value points to, or leads or lags, the entry position of the latest entry of the dirty page buffer to be updated. In some examples x = 0, but other examples could provide a non-zero value of x, e.g. x = +1 or x = -1 .

[0055] Figure 5 illustrates a subset of the SMMLI registers 80 in more detail. In some examples, the SMMLI may support multiple security states (e.g. non-secure, secure and realm security states), and I / O devices may specify an associated security state identifier when issuing I / O transactions to the SMMLI 60. The security states supported by the SMMLI 18 may correspond to corresponding security states supported in the instruction set architecture (ISA) supported by the CPU 20, and the CPU ISA may impose measures which isolate memory regions used to provide data and program code associated with a more secure security state (e.g. the secure or realm security state) from unauthorized access by program code or I / O requests associated with a less secure security state (e.g. the non-secure security state). Similarly, memory regions assigned to the secure security state may be inaccessible to requests associated with the realm security state, and vice versa. Examples of possible architectures for the security state model include the TrustZone® and RME (Realm Management Extension) architectures provided by Arm® Limited. When multiple security states are supported, it may be desirable for address translation and dirty page buffers to be managed separately based on separate control parameters for each security state, and so multiple sets of control registers may be provided, one set per security state. Hence, in this example, the SMMU control registers include multiple sets of registers:

[0056] • register set 82-NS: a set of registers for controlling handling of accesses associated with the non-secure security state;

[0057] • register set 82-S: a set of registers for controlling handling of accesses associated with the secure security state; and

[0058] • register set 82-R: a set of registers for controlling handling of accesses associated with the realm security state;

[0059] It will be appreciated that support for multiple security states is optional, and other examples may only support a single security state and so need not replicate the sets of registers per security state (e.g. in this case register sets 82-S, 82-R could be omitted and register set 82-NS may be used for all accesses). In this example, the set of control registers 82-NS associated with the non-secure security state includes at least the following control registers (it will be appreciated that this is not an exhaustive list and many other control registers could also be provided):

[0060] • SMMU_HDBSS_BASEO: dirty page tracking base address register for identifying a base address of a first non-secure dirty page buffer 0;

[0061] • SMMU_HDBSS_PRODO: producer index register for identifying a producer index value indicative of a next entry to update in the first non-secure dirty page buffer 0;

[0062] • SMMU_HDBSS_BASE1 : dirty page tracking base address register for identifying a base address of a second non-secure dirty page buffer 1 ;

[0063] • SMMU_HDBSS_PROD1 : producer index register for identifying a producer index value indicative of a next entry to update in the second non-secure dirty page buffer 1 ;

[0064] • SMMU_STRTAB_BASE: stream table base address register identifying a base address of a stream table (stored in memory) which provides stream table entries each identifying translation control information for controlling address translations for a corresponding address translation context.

[0065] If additional security states are supported, then each of these registers SMMU_[x] in set 82-NS has corresponding versions SMMU_S_[x] and SMMU_R_[x] in the register sets 82-S, 82-R for the other security states (in this example, those other security states being secure and realm security states respectively). When handling an I / O transaction associated with a given security state, the transaction is processed based on the control information obtained from the set of control registers corresponding to the given security state.

[0066] The control registers shown in Figure 5 may be programmable by software executing on the CPU 20. For example, the SMMU control registers 80 may be memory-mapped registers and software executing on the CPU 20 may program the information in the registers 80 by executing store instructions which specify, as the store target address, an address mapped directly to the SMMU control register 80 to be updated and / or an address mapped to a programming interface register or command queue structure used to control the update to the SMMU control register 80. Hence, there can be a variety of ways in which the programming interface for the SMMU control register 80 could be implemented, but in general software on the CPU 20 has a mechanism by which it can request updates to the SMMU control registers 80.

[0067] Figure 6 illustrates in more detail contents of the SMMU registers 80 shown in Figure 5. For conciseness, Figure 6 illustrates only one of the respective sets 82 of registers for a given security state (where the notation “(*_)” in the register names shown in Figure 6 denotes that this could refer to any of the respective versions of these registers provided per security state). As shown in Figure 5, the SMMU supports a pair of alternate dirty page buffers for each security state - buffer 0 and buffer 1 . In Figure 6, the base address and producer index registers for the pair of buffers are shown more concisely as a single base address register 84 and producer index register 86 (since each of the respective versions of the registers for buffer 0 and buffer 1 have the same contents). Hence, in the notation SMMU_(*_)HDBSS_BASEn and SMMU_(*_)HDBSS_PRODn in Figure 6, n=0 or n=1 and both alternate versions of the BASE / PROD registers 84, 86 have the same contents as shown in Figure 6.

[0068] As shown in Figure 6, each dirty page tracking base address register 84 (SMMU_(*_)HDBSS_BASEn) includes at least:

[0069] • a valid flag V indicating whether the corresponding dirty page buffer is valid (the valid flag V is updatable on request by software, to request changes in which buffer is active);

[0070] • the base address BADDR of the corresponding dirty page buffer, and

[0071] • a size parameter SZ indicating a maximum size for the buffer (which can be used, in conjunction with the INDEX field of the producer index register 86, to detect when the buffer becomes full). The SMMLI 60 may support a variety of size options for the buffer, so the size parameter SZ is encoded to select from a range of options (e.g. any one or more of the following sizes could be supported, with corresponding encodings of the size parameter being allocated to represent each option: 4KB, 8KB, 16KB, 32KB, 64KB, 128KB, 256KB, 512KB, 1MB, 2MB, 4MB, 8MB, 16MB, 32MB, 64MB).

[0072] Each producer index register 86 (SMMU_(*_)HDBSS_PRODn) includes at least:

[0073] • a valid acknowledgement flag VACK settable by hardware to acknowledge to software changes of buffer active status requested by software;

[0074] • an error field ERR for indicating error codes, to inform software if an error has occurred and if so the type of error that occurred. For example, possible reasons for errors could be that the memory access request issued to attempt to write a page address to the dirty page buffer has encountered an address translation fault, or that a given dirty page tracking buffer has become full and there are no remaining valid buffers which could be updated in hardware (e.g. because software is still processing the other buffer and so has set the buffer as invalid).

[0075] • an index field INDEX for indicating the producer index value which is combined with the base address BADDR to obtain an address of the entry of the dirty page buffer to be updated next.

[0076] Each security state has two buffers: buffer 0 and buffer 1 , and a maximum of one of these buffers can be the active buffer at any given time. In this example, which buffer is active is controlled based on the V and VACK indications in the respective registers SMMU_(*_)HDBSS_BASEO, SMMU_(*_)HDBSS_BASE1 , SMMU_(*_)HDBSS_PRODO, SMMU_(*_)HDBSS_PROD1 .

[0077] A given buffer n is valid if both its V and VACK fields are equal to 1. A given buffer is invalid if both its V and VACK fields are equal to 0. If the given buffer’s V and VACK fields do not match, the buffer is in a transitional state during a change of buffer validity.

[0078] If none of the pair of buffers 0 and 1 for a given security state is valid, then there is no active buffer for that security state. If one of the pair of buffers 0 and 1 for a given security state is valid and the other is invalid, then the valid buffer is the currently active buffer and the invalid buffer is inactive. Here, an active buffer is the buffer to which the dirty page tracking circuitry 90 is able to write indications of addresses of pages which have transitioned from clean to dirty. The inactive buffer is not updated by the hardware of the dirty page tracking circuitry, although the inactive buffer may be processed by software executing on a CPU 20 to parse dirty page information previously written to that buffer when that buffer was previously active.

[0079] If both buffers 0 and 1 for a given security state are currently indicated as valid, then the hardware selects which buffer is active based on a hardware-maintained value (not exposed as a software-accessible register). Initially when software configures the dirty page tracking circuitry to start tracking dirty pages for a given translation context, the software sets both buffers to be valid by setting the corresponding valid flags V to 1. Initially, the dirty page tracking circuitry sets buffer 0 as the active buffer.

[0080] If a given buffer becomes full (detected when the producer index value INDEX in the corresponding SMMU_(*_)HDBSS_PRODn register reaches the size limit defined by the corresponding size parameter SZ in SMMU_(*_)HDBSS_BASEn), then if the other buffer 1 is indicated as valid by its corresponding valid flag V in SMMU_(*_)HDBSS_BASEn, the dirty page tracking circuitry 90 switches the active buffer to be the other buffer and can continue tracking dirty pages in buffer 1 (on switching active buffer due to a hardware-triggered detection of the buffer becoming full, the dirty page tracking circuitry 90 can also signal an interrupt to notify software that buffer 0 has become full and can be processed). If a buffer becomes full but the other buffer is indicated as invalid by its valid flag V, then an error is signalled using the ERR field of the currently active buffer that became full.

[0081] Software is able to trigger transitions in buffer validity by programming the buffer’s V field in the corresponding base address register SMMU_(*_)HDBSS_BASEn. When software changes the value of the V field, the dirty page tracking hardware 90 of the SMMU 60 responds by setting the corresponding VACK field in SMMU_(*_)HDBSS_PRODn to match the new value of the V field in SMMU_(*_)HDBSS_BASEn. The dirty page tracking circuitry 90 delays transitioning SMMU_(*_)HDBSS_PRODn.VACK to V=0 until it has completed any outstanding updates to buffer n that were in flight at the time of detecting the software-triggered transition of this buffer from valid to invalid. The software request to invalidate buffer n may be an indication that software now wishes to start processing the indications of dirty pages tracked in the buffer, so hardware should stop updating that buffer and switch to the opposite buffer being active, but to ensure that no outstanding in flight requests continue to update the buffer being transitioned to invalid after software starts parsing information from the buffer, the acknowledgement information VACK from hardware is provided, to allow hardware to defer acknowledging the change in buffer validity until hardware has completed outstanding buffer updates. The software may be programmed to check for the update to VACK before starting to access the newly invalid buffer. In the case of a software-triggered transition of buffer validity from invalid (V=0) to valid (V=1), there is no need for the dirty page tracking circuitry 90 to delay acknowledging the transition, since in this case there would be no outstanding hardware-triggered updates to that buffer. This transition may occur once software has finished processing the contents of a given buffer and now wishes to make that buffer available for hardware updates once more (at this point software may then switch to processing the other buffer).

[0082] While Figure 6 shows a particular layout of the control parameters SZ, ERR, V, VACK within spare portions of the registers used to provide the base address and index values, other examples could record the same information in a different layout within a set of control registers, so it is not essential for the control information shown to be grouped into distinct registers in the particular combinations shown. For example, the error field ERR could be designated in its own register separate from registers 84, 86, or the size field could be moved to the producer index register 86.

[0083] Figure 6 also shows an example of the stream table 130 referenced by the base address in the stream table base register (SMMU_(*_)STRTAB_BASE) 88 for a given security state. The base address in the stream table base register 88 identifies the start address of a stream table structure 130 stored in the memory system. The stream table is organised as a list of stream table entries (STEs) 132, each associated with a corresponding stream identifier which can be used to derive an offset of the address of that stream table entry relative to the base address from register 88. Each stream table entry 132 provides translation control information corresponding to a particular translation context. A given I / O memory system transaction received from devices 40, 50 specifies a stream identifier used to select which stream table entry 132 is used for handling translations for that transaction. The translation control information specified in the stream table entry 132 can include parameters such as a base address of the translation table structure used to provide page table information (address mappings and access permissions), control parameters for defining the structure of the page table (e.g. page size and / or nested structure of a multi-level table structure), etc. As shown in Figure 6, each stream table entry may also specify a dirty page tracking enable value 134 (S2HDBSS) which selects whether dirty page tracking is enabled or disabled for I / O transactions associated with the corresponding translation context. Hence, at step 102 of Figure 4, the dirty page tracking circuitry 90 may determine whether dirty page tracking is enabled, by checking the dirty page tracking enable value 134 of the stream table entry which corresponds to the address translation context identified for the I / O memory system transaction being processed. Information obtained from the stream table 130 may be cached at the IOMMU 60, so it is not essential for the dirty page tracking circuitry 90 to access the stream table structure 130 in memory each time the dirty page tracking enable value 134 or other stream table information is needed.

[0084] Figure 7 shows use of a pair of alternate dirty page buffer structures 150-0, 150-1 associated with a given security state. The base address registers 84-0, 84-1 for the given security state identify the size SZ and base address BADDR of each dirty page buffer structure 150-0, 150-1. The producer index registers 86-0, 86-1 associated with each buffer provide index values (INDEX) used to index into the next buffer entry to be updated in the corresponding buffer. The buffers are memory-based data structures updated by the dirty page tracking circuitry 90 of the SMMII 60 issuing a write memory system request to the memory system (via interconnect 10) to cause data in memory representing the buffer to be updated. At a given time, if dirty page tracking is enabled, a maximum of one of the pair of buffers is active and the active buffer is updated with page addresses (defined in the intermediate address space) of pages which are detected as transitioning from clean to dirty. The other buffer is inactive and can be processed by software to parse previously detected dirty pages.

[0085] Figure 8 illustrates steps for controlling transitions of which buffer is active. As shown in Figure 8, the SMMII 60 supports both hardware-triggered and software-triggered changes of which buffer is active. While Figure 8 shows steps performed in a given sequential order, it will be appreciated that the steps for checking for software-triggered and hardware-triggered dirty page buffer switch events could be performed in parallel, or the steps could be performed in a different order from that shown in Figure 8.

[0086] At step 200, the dirty page tracking circuitry 90 detects whether a hardware-triggered dirty page buffer switch event has occurred. For example, the hardware-triggered dirty page buffer switch event may be detected if the currently active dirty page buffer is detected as being full. For example, the currently active dirty page buffer may be detected as full if the index indicated by the producer index register 86-n for the currently active dirty page buffer is detected as reaching or exceeding the maximum permitted value indicated by the size parameter SZ of the corresponding base address register 84-n. In response to the hardware- triggered dirty page buffer switch event, at step 202 the dirty page tracking circuitry switches which of the pair of dirty page buffers 150-0, 150-1 is active (assuming the buffer being switched to is currently indicated as valid by its validity flag V and is not itself full already - if the next buffer is currently invalid or is already full then dirty page tracking is halted and an error can be signalled using the ERR field associated with the current active buffer).

[0087] Meanwhile, if at step 204 a software-triggered dirty page buffer switch event is detected, triggered by a software request to update the dirty page buffer validity information (V flag in the base address register of the buffer which is currently active), then at step 208 the dirty page tracking circuitry 90 detects whether there are any outstanding updates to complete for the current active page buffer, and if so waits for those accesses to complete, before updating the software- accessible acknowledgement indication (VACK flag for the outgoing active buffer) at step 210. At step 202, the dirty page tracking circuitry 90 switches which dirty page buffer is active.

[0088] If neither hardware-triggered nor software- triggered buffer switch event has occurred, then at step 206 there is no change to which dirty page buffer is active. It will be appreciated that the particular examples of control state shown in Figure 6 is just one particular implementation, and the control state could be implemented in a different way for other examples. Nevertheless, for supporting live migration use cases (unlike purely hardwaremanaged multi-buffer arrangements), it is useful to offer software the ability to trigger a switch of which buffer is active for dirty page tracking, so that the live migration software can signal to the SMMII hardware when it wants to start processing the contents of the current active buffer.

[0089] Hence, the provision of two (or more) buffers associated with the same security state, for alternately tracking addresses of dirty pages caused to become dirty by I / O memory system transactions, can be helpful for supporting live migration because it means that, even though I / O devices 40, 50 cannot be interrupted, it is possible to halt updates to a given buffer to allow the buffer to be handed over to software for processing, while hardware-triggered updates of dirty page information continue in the other buffer following a switch of which buffer is active.

[0090] Concepts described herein may be embodied in computer-readable code for fabrication of an apparatus that embodies the described concepts. For example, the computer-readable code can be used at one or more stages of a semiconductor design and fabrication process, including an electronic design automation (EDA) stage, to fabricate an integrated circuit comprising the apparatus embodying the concepts. The above computer-readable code may additionally or alternatively enable the definition, modelling, simulation, verification and / or testing of an apparatus embodying the concepts described herein.

[0091] For example, the computer-readable code for fabrication of an apparatus embodying the concepts described herein can be embodied in code defining a hardware description language (HDL) representation of the concepts. For example, the code may define a register-transfer-level (RTL) abstraction of one or more logic circuits for defining an apparatus embodying the concepts. The code may define a HDL representation of the one or more logic circuits embodying the apparatus in Verilog, SystemVerilog, Chisel, or VHDL (Very High-Speed Integrated Circuit Hardware Description Language) as well as intermediate representations such as FIRRTL. Computer-readable code may provide definitions embodying the concept using system-level modelling languages such as SystemC and SystemVerilog or other behavioural representations of the concepts that can be interpreted by a computer to enable simulation, functional and / or formal verification, and testing of the concepts.

[0092] Additionally or alternatively, the computer-readable code may define a low-level description of integrated circuit components that embody concepts described herein, such as one or more netlists or integrated circuit layout definitions, including representations such as GDSII. The one or more netlists or other computer-readable representation of integrated circuit components may be generated by applying one or more logic synthesis processes to an RTL representation to generate definitions for use in fabrication of an apparatus embodying the invention. Alternatively or additionally, the one or more logic synthesis processes can generate from the computer-readable code a bitstream to be loaded into a field programmable gate array (FPGA) to configure the FPGA to embody the described concepts. The FPGA may be deployed for the purposes of verification and test of the concepts prior to fabrication in an integrated circuit or the FPGA may be deployed in a product directly.

[0093] The computer-readable code may comprise a mix of code representations for fabrication of an apparatus, for example including a mix of one or more of an RTL representation, a netlist representation, or another computer-readable definition to be used in a semiconductor design and fabrication process to fabricate an apparatus embodying the invention. Alternatively or additionally, the concept may be defined in a combination of a computer-readable definition to be used in a semiconductor design and fabrication process to fabricate an apparatus and computer- readable code defining instructions which are to be executed by the defined apparatus once fabricated.

[0094] Such computer-readable code can be disposed in any known transitory computer- readable medium (such as wired or wireless transmission of code over a network) or non- transitory computer-readable medium such as semiconductor, magnetic disk, or optical disc. An integrated circuit fabricated using the computer-readable code may comprise components such as one or more of a central processing unit, graphics processing unit, neural processing unit, digital signal processor or other components that individually or collectively embody the concept.

[0095] Figure 9 illustrates a simulator implementation that may be used. Whilst the earlier described embodiments implement the present invention in terms of apparatus and methods for operating specific processing hardware supporting the techniques concerned, it is also possible to provide an instruction execution environment in accordance with the embodiments described herein which is implemented through the use of a computer program. Such computer programs are often referred to as simulators, insofar as they provide a software based implementation of a hardware architecture. Varieties of simulator computer programs include emulators, virtual machines, models, and binary translators, including dynamic binary translators. Typically, a simulator implementation may run on a host processor 730, optionally running a host operating system 720, supporting the simulator program 710. In some arrangements, there may be multiple layers of simulation between the hardware and the provided instruction execution environment, and / or multiple distinct instruction execution environments provided on the same host processor. Historically, powerful processors have been required to provide simulator implementations which execute at a reasonable speed, but such an approach may be justified in certain circumstances, such as when there is a desire to run code native to another processor for compatibility or re-use reasons. For example, the simulator implementation may provide an instruction execution environment with additional functionality which is not supported by the host processor hardware, or provide an instruction execution environment typically associated with a different hardware architecture. An overview of simulation is given in “Some Efficient Architecture Simulation Techniques”, Robert Bedichek, Winter 1990 IISENIX Conference, Pages 53 - 63. To the extent that embodiments have previously been described with reference to particular hardware constructs or features, in a simulated embodiment, equivalent functionality may be provided by suitable software constructs or features. For example, particular circuitry may be implemented in a simulated embodiment as computer program logic. Similarly, memory hardware, such as a register or cache, may be implemented in a simulated embodiment as a software data structure. In arrangements where one or more of the hardware elements referenced in the previously described embodiments are present on the host hardware (for example, host processor 730), some simulated embodiments may make use of the host hardware, where suitable.

[0096] The simulator program 710 may be stored on a computer-readable storage medium (which may be a non-transitory medium), and provides a program interface (instruction execution environment) to the target code 700 (which may include applications, operating systems and a hypervisor) which is the same as the interface of the hardware architecture being modelled by the simulator program 710. Thus, the program instructions of the target code 700 described above, may be executed from within the instruction execution environment using the simulator program 710, so that a host computer 730 which does not actually have the hardware features of the apparatus 2 discussed above can emulate these features.

[0097] In the context of a simulation program 710 which simulates a system comprising an SMMLI 60 as discussed earlier, the simulator code 710 may include address translation program logic 712, register simulating program logic 714 and dirty page tracking program logic 716, which controls the host hardware 730 to emulate the hardware functionality provided by the address translation circuitry 70, SMMLI registers 80 and dirty page tracking circuitry 90 described earlier for the SMMLI 60 in Figure 3. Hence, when simulated I / O transactions are detected, the dirty page tracking program logic 716 manages updates to dirty page buffer structures in simulated memory (mapped to memory of the host 730), based on register state associated with registers simulated by the register simulating program logic 714 which mirrors the control register state of the SMMLI registers 80 described above. Those control registers 80 may be simulated by mapping them to corresponding storage locations (e.g. host memory) provided by the host hardware 730.

[0098] In the present application, the words “configured to...” are used to mean that an element of an apparatus has a configuration able to carry out the defined operation. In this context, a “configuration” means an arrangement or manner of interconnection of hardware or software. For example, the apparatus may have dedicated hardware which provides the defined operation, or a processor or other processing device may be programmed to perform the function. “Configured to” does not imply that the apparatus element needs to be changed in any way in order to provide the defined operation.

[0099] In the present application, lists of features preceded with the phrase “at least one of’ mean that any one or more of those features can be provided either individually or in combination. For example, “at least one of: [A], [B] and [C]” encompasses any of the following options: A alone (without B or C), B alone (without A or C), C alone (without A or B), A and B in combination (without C), A and C in combination (without B), B and C in combination (without A), or A, B and C in combination. Although illustrative embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications can be effected therein by one skilled in the art without departing from the scope of the invention as defined by the appended claims.

Claims

CLAIMS1. An input / output memory management unit (IOMMU), the IOMMU comprising: address translation circuitry to control address translation in response to input / output memory system transactions issued by one or more input / output devices; at least one dirty page tracking base address register to store a base address of a dirty page buffer structure for tracking addresses of dirty pages of address space; and dirty page tracking circuitry to: detect whether an input / output memory system transaction associated with a given translation context for which dirty page tracking is enabled causes a transition of a given page of address space from clean to dirty, and in response to detecting the input / output memory system transaction associated with the given translation context for which dirty page tracking is enabled causing a transition of the given page from clean to dirty, issue a write memory access request requesting that a location within a selected dirty page buffer structure, identified using the base address indicated in a selected dirty page tracking base address register, is updated to identify a page address of the given page.

2. The IOMMU according to claim 1 , comprising at least one producer index register to store a producer index value indicative of a dirty page buffer entry to be updated next; in which: the dirty page tracking circuitry is configured to identify said location within the selected dirty page buffer structure using the base address indicated in the selected dirty page tracking base address register and the producer index value indicated in a corresponding producer index register corresponding to the selected dirty page tracking base address register.

3. The IOMMU according to any of claims 1 and 2, comprising a plurality of dirty page tracking base address registers to store base addresses of a plurality of dirty page buffer structures; in which: the dirty page tracking circuitry is configured to select, as the selected dirty page buffer structure, an active dirty page buffer structure indicated by dirty page buffer status information as being currently in use by the dirty page tracking circuitry.

4. The IOMMU according to claim 3, in which the dirty page tracking circuitry is responsive to occurrence of a dirty page buffer switch event, to switch which of at least two dirty page buffer structures is the active dirty page buffer structure.

5. The IOMMU according to claim 4, in which the dirty page tracking circuitry is configured to support a software-triggered dirty page buffer switch event triggered by a software request to switch which of the at least two dirty page buffer structures is the active dirty page buffer structure.

6. The IOMMU according to claim 5, in which the software request comprises a request to update dirty page buffer validity information associated with a given dirty page buffer structure which is currently the active dirty page buffer structure, to indicate that the given dirty page buffer structure should become invalid.

7. The IOMMU according to any of claims 5 and 6, in which in response to the software request, the dirty page tracking circuitry is configured to: complete any outstanding updates to the given dirty page buffer structure; and in response to completion of any outstanding updates to the given dirty page buffer structure, update a software-accessible acknowledgement indication; and switch which of said at least two dirty page buffer structures is selected as the active dirty page buffer structure.

8. The IOMMU according to any of claims 4 to 7, in which the dirty page tracking circuitry is configured to support a hardware-triggered dirty page buffer switch event.

9. The IOMMU according to claim 8, in which the hardware-triggered dirty page buffer switch event comprises detection that the active dirty page buffer structure has become full.

10. The IOMMU according to claim 9, in which the dirty page tracking circuitry is configured to detect whether the active dirty page buffer structure has become full based on: a producer index value indicative of a next entry to update in the active dirty page buffer structure; and a software-programmable size parameter indicating a maximum size of the active dirty page buffer structure.

11. The IOMMU according to any of claims 4 to 10, in which the dirty page tracking circuitry is configured to support both: a software-triggered dirty page buffer switch event triggered by a software request to switch which of the at least two dirty page buffer structures is the active dirty page buffer structure; and a hardware-triggered dirty page buffer switch event.

12. The IOMMU according to any preceding claim, in which the dirty page buffer structure has a configurable size defined by a software-programmable size parameter.

13. The IOMMU according to any preceding claim, in which the dirty page tracking circuitry is configured to determine whether dirty page tracking is enabled for the given translation context based on a stream table entry defining translation control information for the given translation context.

14. The IOMMU according to any preceding claim, comprising a plurality of dirty page tracking base address registers to store base addresses of respective dirty page buffer structures corresponding to different security states; in which: the dirty page tracking circuitry is configured to select the selected dirty page buffer structure based on an associated security state associated with the input / output memory system transaction.

15. The IOMMU according to any preceding claim, in which: the address translation circuitry is configured to support two-stage address translation in which translation of a virtual address to a physical address depends on first-stage address translation information mapping the virtual address in a virtual address space to an intermediate address in an intermediate address space and second-stage address translation information mapping the intermediate address in the intermediate address space to the physical address in a physical address space; and the dirty page tracking circuitry is configured to maintain the dirty page buffer structure to identify the dirty pages of address space using intermediate page addresses of the intermediate address space.

16. The IOMMU according to any preceding claim, in which the at least one dirty page tracking base address register is separate from a base address structure for identifying a base address of a translation table structure used to provide address translation mapping information for the address translation controlled by the address translation circuitry.

17. The IOMMU according to any preceding claim, in which the dirty page tracking circuitry is configured to maintain, as the dirty page buffer structure, a list of page addresses of the dirty pages of address space.

18. Computer-readable code for fabrication of an IOMMU according to any of claims 1 to 17.

19. A storage medium storing the computer-readable code of claim 18.

20. A method comprising: controlling address translation in response to input / output memory system transactions issued by one or more input / output devices; detecting whether an input / output memory system transaction associated with a given translation context for which dirty page tracking is enabled causes a transition of a given page of address space from clean to dirty, and in response to detecting the input / output memory system transaction associated with the given translation context for which dirty page tracking is enabled causing a transition of the given page from clean to dirty, issuing a write memory access request requesting that a location within a selected dirty page buffer structure, identified using a base address indicated in a selected dirty page tracking base address register, is updated to identify a page address of the given page.

21. A computer program comprising instructions which, when executed on a host data processing apparatus, control the host data processing apparatus to simulate an input / output memory management unit (IOMMU), the computer program comprising: address translation program logic to control the host data processing apparatus to perform address translation in response to simulated input / output memory system transactions issued by one or more simulated input / output devices; and dirty page tracking program logic to: detect whether a simulated input / output memory system transaction associated with a given translation context for which dirty page tracking is enabled causes a transition of a given page of simulated address space from clean to dirty, and in response to detecting the simulated input / output memory system transaction associated with the given translation context for which dirty page tracking is enabled causing a transition of the given page from clean to dirty, issue a write memory access request requesting that a location within a selected dirty page buffer structure, identified using a base address indicated in a selected simulated dirty page tracking base address register, is updated to identify a page address of the given page.

22. A storage medium storing the computer program of claim 21.

Images