Bandwidth utilization limitations

By leveraging the synergistic effect of request processing components and regulating circuits, and controlling bandwidth utilization based on transaction feedback signals, the problem of bandwidth imbalance between processes is solved, thereby improving fairness and system efficiency.

CN123125571APending Publication Date: 2026-11-17ARM LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202580029580.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-04-26
Filing Date
2025-03-18
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Storage transactions requested by processes running on processing elements can lead to uneven bandwidth utilization, affecting the availability of other processes. Existing technologies struggle to effectively control and regulate this.

Method used

By using request processing elements and adjustment circuits, bandwidth utilization is controlled based on transaction feedback signals. Predefined limits are set and dynamically adjusted according to resource utilization parameters to ensure fair share and improve total throughput.

Benefits of technology

It achieves fairness in bandwidth utilization between processes and improves system efficiency, avoiding resource shortages or over-utilization caused by a single process monopolizing bandwidth.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125571A_ABST
    Figure CN123125571A_ABST
Patent Text Reader

Abstract

An apparatus, a method, and a computer program are provided. The apparatus includes a request processing element that issues a storage transaction in response to an access request from a process associated with an identifier. The apparatus also includes regulating circuitry that controls the bandwidth available for storage transactions requested by the process associated with the identifier. The regulating circuitry is configured to control the bandwidth based on a transaction feedback signal issued by circuitry other than the request processing element and indicating resource utilization parameters: when the resource utilization parameters meet resource utilization conditions, control is applied to limit utilization to a predefined limit assigned to the identifier; and when the resource utilization parameters do not meet resource utilization conditions, the control is modified based on the resource utilization parameters.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to data processing. More specifically, this invention relates to an apparatus, method, and computer program. Background Technology

[0002] Memory transactions requested by processes running on the processing element utilize bandwidth. Some processes may require high bandwidth utilization, which could reduce bandwidth availability for other processes attempting to issue memory requests. Summary of the Invention

[0003] According to some examples of the present technology, an apparatus is provided, the apparatus comprising:

[0004] A request processing element configured to issue a storage transaction in response to a storage access request from a process associated with an identifier; and

[0005] A regulating circuit configured to control bandwidth utilization available for storage transactions requested by a process associated with an identifier, wherein, when operating in at least one mode, the regulating circuit is configured to control bandwidth utilization based on a transaction feedback signal issued by circuitry other than the request processing element and indicating resource utilization parameters:

[0006] When resource utilization parameters meet resource utilization conditions, application control limits bandwidth utilization to predefined limits assigned to the identifier; and

[0007] When the resource utilization parameters do not meet the resource utilization conditions, the control application is modified based on the resource utilization parameters.

[0008] Based on some examples of this technology, a method is provided that includes:

[0009] Using a request processing element, a storage transaction is issued in response to a storage access request from a process running on the request processing element, the process being associated with an identifier; and

[0010] Control is available for bandwidth utilization of storage transactions requested by a process associated with an identifier, wherein, when operating in at least one mode, the adjustment includes controlling bandwidth utilization based on a transaction feedback signal issued by circuitry other than the request processing element and indicating resource utilization parameters:

[0011] When resource utilization parameters meet resource utilization conditions, application control limits bandwidth utilization to predefined limits assigned to the identifier; and

[0012] When the resource utilization parameters do not meet the resource utilization conditions, the control application is modified based on the resource utilization parameters.

[0013] According to some examples of the present technology, a computer program is provided for controlling a host data processing device to provide an instruction execution environment, the computer program comprising:

[0014] Request processing element program logic, configured to issue a storage transaction in response to a storage access request from a process associated with an identifier; and

[0015] A regulating program logic configured to control bandwidth utilization available for storage transactions requested by a process associated with an identifier, wherein, when operating in at least one mode, the regulating program logic is configured to control bandwidth utilization based on transaction feedback signals issued by program logic other than the request processing element program logic and indicating resource utilization parameters:

[0016] When resource utilization parameters meet resource utilization conditions, application control limits bandwidth utilization to predefined limits assigned to the identifier; and

[0017] When the resource utilization parameters do not meet the resource utilization conditions, the control application is modified based on the resource utilization parameters.

[0018] According to some configurations of this technology, the computer program is stored on a computer-readable storage medium.

[0019] According to some configurations of this technology, the computer-readable storage medium is a non-transitory computer-readable storage medium. Attached Figure Description

[0020] The invention will be further described by way of example only, with reference to the configuration of the invention illustrated in the accompanying drawings, wherein:

[0021] Figure 1 The apparatus is illustrated schematically with some configurations according to the present technology;

[0022] Figure 2 The apparatus is illustrated schematically with some configurations according to the present technology;

[0023] Figure 3 Some configurations of memory systems according to the present technology are illustrated schematically;

[0024] Figure 4 The apparatus is illustrated schematically with some configurations according to the present technology;

[0025] Figure 5 Examples of different software execution environments executed by processing circuitry are shown;

[0026] Figure 6An example of assigning partition identifiers to different software execution environments is shown;

[0027] Figure 7 An example of a control register used to control which partition identifier is assigned for a given memory transaction is shown;

[0028] Figure 8 The apparatus is illustrated schematically with some configurations according to the present technology;

[0029] Figure 9 The sequence of steps performed according to some configurations of this technology is illustrated schematically;

[0030] Figure 10 The apparatus is illustrated schematically with some configurations according to the present technology;

[0031] Figure 11 The sequence of steps performed according to some configurations of this technology is illustrated schematically;

[0032] Figure 12 The apparatus is illustrated schematically with some configurations according to the present technology;

[0033] Figure 13 The sequence of steps performed according to some configurations of this technology is illustrated schematically; and

[0034] Figure 14 The illustrative examples illustrate specific implementations of simulators with some configurations according to this technology. Detailed Implementation

[0035] Before discussing the configuration with reference to the accompanying drawings, the following description of the configuration is provided.

[0036] According to some configurations of the present technology, an apparatus is provided, comprising a request processing element configured to issue a storage transaction in response to a storage access request from a process associated with an identifier. The apparatus also includes regulating circuitry configured to control bandwidth utilization available for storage transactions requested by the process associated with the identifier. When operating in at least one mode, the regulating circuitry is configured to control bandwidth utilization based on a transaction feedback signal issued by circuitry other than the request processing element and indicating resource utilization parameters: when the resource utilization parameters meet resource utilization conditions, control is applied to limit bandwidth utilization to a predefined limit assigned to the identifier; and when the resource utilization parameters do not meet resource utilization conditions, modification is applied to the control based on the resource utilization parameters.

[0037] Controlling bandwidth usage by processes based on identifiers assigned to them can prevent processes assigned to specific identifiers from monopolizing bandwidth availability and, in some cases, prevent other processes from issuing bandwidth-required transactions in a timely manner. This control can be achieved by assigning predefined limits to identifiers, such as limiting the maximum bandwidth usage of the process associated with the identifier or limiting the minimum bandwidth usage provided to the process associated with the identifier. Therefore, the identifiers associated with processes, and thus the transaction requests issued by or on behalf of these processes, are used to control bandwidth availability, rather than necessarily indicating which memory regions a given process can or cannot access.

[0038] The inventors have recognized that, in some use cases, bandwidth utilization restrictions based on predefined limits assigned to identifiers can lead to underutilization of total available bandwidth or overutilization of available bandwidth by some processes. Specifically, when bandwidth utilization is controlled at the level of a processing element, that processing element may not have a complete picture of all bandwidth utilization in the device by the processes associated with the identifier or other processes associated with different identifiers. Therefore, the regulating circuit is configured with at least one operating mode (in some configurations, this operating mode may be the only operating mode, or in other configurations, it may be one of several operating modes), wherein the specific implementation of the control depends on feedback signals provided from circuitry other than the request processing element. In other words, the control of transaction requests by the request processing element can be modified (e.g., altered or influenced) by one or more other circuitry within the device. The feedback signals include resource utilization parameters indicating resource usage in the device. Depending on whether the resource utilization parameters meet resource utilization conditions, the regulating circuit can execute control specified by predefined limits (i.e., when resource utilization conditions are met), or can modify the control (i.e., when resource utilization conditions are not met). Therefore, the control applied when resource utilization conditions are not met is either in addition to or supplemented by the control that limits bandwidth utilization to predefined limits assigned to the identifier. Using transaction feedback signals to modify control based on resource utilization allows the regulation circuitry to ensure that predefined limits are applied under certain system conditions (when resource utilization conditions are met), and that processes receive a fair share of bandwidth availability (as defined by the predefined limits). Furthermore, since the conditions in the device are outside the processing element or otherwise unknown to the processing element, the regulation circuitry can adapt these limits to improve bandwidth utilization. Therefore, the overall transaction throughput can be increased, thereby improving processing efficiency.

[0039] Resource utilization parameters can relate to general resource utilization, such as that resulting from processes associated with one or more different identifiers. However, in some configurations, resource utilization parameters indicate the resource utilization of the processes associated with the identifiers. Therefore, resource utilization parameters can provide an indication of the actual resource utilization of a process based on one or more parameters of the memory system or other processing elements running the processes associated with the identifiers.

[0040] In some configurations, resource utilization parameters include indications of resource utilization other than that of the request processing element. Resource utilization parameters may include a global resource utilization characteristic that indicates the utilization of all resources, including the request processing element and resources other than the request processing element. Alternatively, resource utilization parameters may include resource-specific utilization characteristics that indicate specific utilization of one or more other resources.

[0041] In some configurations, resource utilization parameters include processing element identification information indicating the processing element running the process associated with the identifier, and resource utilization conditions include processing element conditions that must be met when the requesting processing element is the only processing element identified as running the process associated with the identifier. Therefore, if multiple processing elements are identified as running processes associated with the identifier, the processing element conditions are not met. When the requesting processing element is the only processing element running the process associated with the identifier, a predetermined limit can be interpreted as limiting the bandwidth utilization of that process running on that processing element. When multiple processing elements are identified as running processes associated with the identifier, if each of the processing elements is assigned a predefined limit, the total bandwidth theoretically available to the processes assigned to the identifier will be the predefined limit multiplied by the number of processing elements running those processes. Therefore, the processes assigned to the identifier will be able to obtain significantly more bandwidth utilization than the limits assigned to them. Thus, modifications to control based on resource utilization parameters allow for the implementation of fair share policies without prior knowledge of how many processing elements will run the processes assigned to the identifier. Furthermore, if the number of processing elements running the process assigned to the identifier changes dynamically during runtime, the adjustment circuit can respond to this information as indicated in the feedback signal and can adapt to the limit applied to the requesting processing element.

[0042] While processing element identification information can take any form—for example, a single bit indicating whether one or more processing elements are running a process associated with the identifier—in some configurations, the processing element identification information indicates the number of processing elements running the process associated with the identifier, and modifies the limitation of bandwidth utilization to a reduced limit based on the number of processing elements. This reduced limit can be updated during runtime based on the number of processing elements and can be further changed based on one or more other conditions included in the resource utilization parameters.

[0043] In some configurations, the reduction limit is calculated by dividing the bandwidth utilization limit by the number of processing elements running the process associated with the identifier. This division can be a strict mathematical division, where the limit is calculated as A / B, where A equals the bandwidth utilization limit and B equals the number of processing elements. Alternatively, the division can include sharing bandwidth among processing elements. For example, a resource utilization parameter can indicate the number of processing elements and an indication of a bandwidth utilization score, which indicates the score of the total bandwidth utilization of the identifier associated with each processing element. In such configurations, the reduction limit can be segmented based on the score for each processing element. In some configurations, the division can be an approximate division. For example, the number of processing elements, B, can be rounded to the nearest power of 2 to allow the use of shift circuitry to calculate the division. In other words, B is rounded to 2. P Furthermore, the reduction limit is calculated by shifting the bandwidth utilization limit A to the right by P positions. This method avoids the need for a full division calculation and yields an approximate reduction limit suitable for bandwidth utilization control.

[0044] In some configurations, the device includes interconnect circuitry configured to store processing element utilization information indicating the number of processing elements issuing transaction requests associated with identifiers, wherein the interconnect is configured to emit feedback signals indicating the number of processing elements. When a transaction is issued, a processing element can associate the identifier with the transaction and an indication of the processing element requesting the transaction. Including identifiers can allow one or more circuits receiving transactions to implement their own fair share policies associated with identifiers and can be used by the interconnect circuitry to track which processing elements are running processes assigned to identifiers. The processing element utilization information can include a bitmap storing an indication of each processing element that has issued a transaction request with an identifier. This bitmap can be stored in a set-associative memory structure based on an identifier index and having entries for each processing element of the device identifying whether the processing element has issued a transaction request associated with an identifier.

[0045] In some configurations, interconnects are configured to apply an aging mechanism to process element utilization information. For example, the aging mechanism may include recording information indicating a timestamp, provided by a repeating counter, each time a transaction associated with an identifier is received. Once the counter has completed a cycle and reached the same value as the stored timestamp, the information may become invalid.

[0046] In some configurations, the aging mechanism includes storing processing element utilization information on a sliding window. In the event of a transaction receiving an indicator identifier from the processing element within the sliding window, this indicator can be stored by the interconnect circuitry to indicate that the processing element is actively running the process associated with the identifier. In the event that no transaction receives an indicator identifier from the processing element during the sliding window, any indicators stored in the interconnect circuitry can be zeroed out. It will be apparent to those skilled in the art that alternative aging mechanisms can be applied to the processing element utilization information.

[0047] In some configurations, resource utilization parameters include congestion parameters indicating congestion of storage requests, and resource utilization conditions include congestion conditions that must be met when the congestion parameters exceed a congestion threshold. Congestion parameters may be included in resource utilization parameters in addition to, or as an alternative to, processing element utilization information. Congestion parameters may be existing parameters provided by the storage system to indicate the total utilization of the storage buffer. For example, congestion parameters may include a multi-bit signal indicating the congestion level of the storage system, such as no congestion, light congestion, or heavy congestion. A congestion threshold is considered exceeded when the congestion parameters indicate that there is no congestion. Alternatively, a congestion threshold may be considered exceeded only when the congestion parameters indicate heavy congestion.

[0048] In some configurations, the limiting circuitry is configured to allow processing elements to issue requests at a rate greater than a predefined limit when modifications are applied. For example, when a congestion parameter indicates a low congestion level (e.g., no congestion), the regulating circuitry may allow exceeding the predefined limit based on the knowledge that (due to the low congestion level) there is still sufficient bandwidth available for transactions published by processes associated with one or more other identifiers.

[0049] While allowing processing elements to issue requests at rates exceeding predefined limits may include allowing processing elements to issue requests in an unrestricted manner, in some configurations, limiting circuitry is configured to apply soft limits to bandwidth utilization upon application modification. These soft limits allow bandwidth utilization to exceed predefined limits. Soft limits can be software-specified limits that can be edited based on each identifier. Alternatively, soft limits can be specified as global percentage increments that can be applied to predefined limits. In some configurations, multiple soft limits can be provided, and each soft limit can be applied based on different congestion levels. For example, a predefined limit can be applied when congestion is high, a first soft limit can be applied when congestion is low, and a second soft limit (allowing greater bandwidth utilization than the first soft limit) can be applied when there is no congestion identified by resource utilization parameters.

[0050] In some configurations, utilization parameters are issued by the storage hierarchy. Resource utilization parameters can be issued by one or more storage levels in the storage hierarchy. For example, resource utilization parameters can be issued by one or more levels of cache and / or from main system memory (e.g., DRAM).

[0051] In some configurations, the device includes one or more software-accessible registers, in which predefined limits are stored. The software-accessible registers can be software-configurable registers, which can be configured by software operating at one or more different privilege levels. For example, a software-accessible register can be configured by software with a privilege level greater than a threshold privilege level.

[0052] In some configurations, an identifier is one of multiple identifiers, each of which can be assigned to one or more processes, and predefined limits are set based on each identifier. These predefined limits can be set as part of the architectural state associated with the currently executing process and can be loaded into one or more software-accessible registers as part of the process's execution. When the processor performs a context switch from the current process to a different process that can be associated with a different identifier, the predefined limits associated with the different identifier can be loaded instead of the predefined limits associated with the current process.

[0053] While at least one mode may be the only mode in which the regulation circuitry can operate, in some configurations, the request processing element can operate in another mode in which the regulation circuitry is configured to limit bandwidth utilization to a predefined limit independently of the transaction feedback signal. In other words, the regulation circuitry can operate in a mode in which the predefined limit associated with the identifier is strictly enforced for the request processing element independently of the feedback signal. The operating mode can be controlled by software operating at a higher privilege level; for example, a hypervisor or operating system can be assigned a sufficiently high privilege level to control the operating mode.

[0054] In some configurations, the request processing element is configured to suspend the execution of a process in response to the satisfaction of one or more limits. In some configurations, the processing element can respond to the suspension by context switching to a different process with a different identifier (e.g., an identifier that has not yet reached a predefined limit associated with that identifier). Alternatively, the processing element can remain suspended until the regulation circuitry identifies that the bandwidth utilization associated with the identifier has decreased, or the regulation circuitry modifies the control (e.g., due to a change in resource utilization parameters) so that transaction requests can still be issued by (or on behalf of) the processing element.

[0055] In some configurations, process-associated identifiers are defined in software-configurable registers. These registers can be dedicated registers configured to store identifiers, or they can be shared registers that also share information about predefined limits for the identifiers. Software-configurable registers can be configured by software with a privilege level greater than a threshold privilege level. For example, they can be configured by a hypervisor or an operating system operating at a higher privilege level than user applications.

[0056] A specific configuration will now be described with reference to the accompanying drawings.

[0057] Figure 1 An apparatus 100 according to some configurations of the present technology is illustrated. The apparatus 100 includes a processing element 102 and a regulating circuit 104. The processing element includes, for example, the following regarding... Figure 4 The described processing circuitry is configured to execute a sequence of operations associated with a process, defined by an identifier. Processing element 102 responds to several types of instructions, such as load and store instructions, to trigger a transaction request to be issued to storage circuitry. Regulation circuitry 104 is configured to control bandwidth utilization available for transactions requested by processing element 102 based on the identifier assigned to the process executing on processing element 102. This control is based on a transaction feedback signal received from circuitry other than processing element 102. This transaction feedback signal indicates resource utilization parameters that indicate resource utilization. Regulation circuitry 104 determines whether the transaction feedback signal satisfies resource utilization conditions. When regulation circuitry 104 determines that resource utilization conditions are met, it limits the bandwidth utilization of processing element 102 to a predefined limit associated with the identifier assigned to the process. When regulation circuitry 104 determines that resource utilization conditions are not met, it applies modifications to the control based on the resource utilization parameters.

[0058] Figure 2 Examples of apparatus 2 according to some configurations of the present technology are schematically illustrated. Apparatus 2 includes N processing clusters 4 (N is 1 or more), wherein each processing cluster includes one or more processing elements 6, such as CPUs (Central Processing Units) or GPUs (Graphics Processing Units). Each processing element 6 may have at least one cache, such as a Level 1 data cache 8, a Level 1 instruction cache 10, and a shared Level 2 cache 12. It should be understood that this is only one example of a possible cache hierarchy structure, and other cache arrangements may be used. Processing elements 6 within the same cluster are coupled by cluster interconnects 14. The cluster interconnects 14 may have a cluster cache 16 for caching data accessible by any processing element.

[0059] The system-on-chip (SoC) interconnect 18 is coupled to the N clusters and any other requester devices 22 (such as a display controller or a direct memory access (DMA) controller). The SoC interconnect may have a system cache 20 for caching data accessible by any requester connected to it. The SoC interconnect 18 controls the consistency between the respective caches 8, 10, 12, 16, and 20 according to any known consistency protocol. The SoC interconnect is also coupled to one or more memory controllers 24, each controlling access to a corresponding memory 25 (such as DRAM or SRAM). The SoC interconnect 18 may also route transactions to other completer devices, such as cryptographic units for providing encryption / decryption functionality.

[0060] Therefore, the data processing system 2 includes a memory system for storing data and providing access to the data in response to transactions initiated by the processing element 6 and other requester devices 22. Caches 8, 10, 12, 16, and 20, interconnects 14 and 18, the memory controller 24, and the memory device 25 can each be considered as components of this memory system. Other examples of memory system components may include a memory management unit or translation buffer (within the processing element 6 itself or further down within the system interconnect 18 or another part of the memory system) for translating memory addresses for accessing memory, and thus may also be considered part of the memory system. Typically, memory system components may include any components of the data processing system used to service memory transactions to access memory data or to control the processing of those memory transactions.

[0061] The memory system may have various resources for processing memory transactions. For example, caches 8, 10, 12, 16, and 20 may have storage capacity available for caching data required by a given software execution environment executing on one of the processing elements 6, to provide faster access to data or instructions than if they had to be retrieved from main memory 25. Similarly, the MMU / TLB may have capacity available for caching address-translated data. Furthermore, interconnects 14 and 18, the memory controller 24, and the memory device 25 may each have a certain amount of bandwidth available for processing memory transactions.

[0062] When multiple software execution environments (QEs) running on processing element 6 share access to the memory system, it may be desirable to prevent one QE from using more resources than its fair share, thereby preventing other QEs from experiencing a performance penalty. This can be particularly important for data center (server) applications, where there is a need to improve data center server utilization by increasing the number of independent software processes interacting with a given amount of memory capacity to reduce the increase in capital expenditure. However, tail latency targets for web applications still need to be met, and it is undesirable if a process running on the server monopolizes memory system resources to a degree that is detrimental to other processes. Similarly, for networking applications, it is increasingly common to combine multiple functions that were previously on separate SoCs onto a single SoC. This again leads to the desire to limit the performance interactions between QEs, and to monitor how these QEs need to allow those independent processes access to shared memory while limiting performance interactions.

[0063] Figure 3 This illustration demonstrates an example of controlling the allocation of memory system resources based on the software execution environment that issues the corresponding memory transaction. In this context, the software execution environment can be any process or part of a process executed by a processing element within a data processing system. For example, a software execution environment can include an application, a guest operating system or virtual machine, a host operating system or hypervisor, a security monitor program for managing different security states of the system, or a sub-part of any of these types of processes (e.g., a single virtual machine can have different parts that are considered separate software execution environments). Figure 3 As shown, a specific partition identifier (PartID) 30 can be assigned to each software execution environment. This partition identifier, along with the memory transactions associated with that software execution environment, is transmitted to the memory system component. The partition identifier is an example of an identifier.

[0064] Within a memory system component, resource allocation or contention resolution operations can be controlled based on one of several memory system component parameter groups selected according to a partition identifier. For example, ... Figure 3 As shown, each software execution environment can be assigned an allocation threshold (an example of a predefined limit), which represents the maximum amount of cache capacity that can be allocated for the data / instructions associated with that software execution environment. The relevant allocation threshold is selected based on the partition identifier associated with a given transaction when servicing that transaction. For example, in Figure 3 In this context, a transaction associated with partition identifier 0 can allocate up to 50% of the cache's storage capacity, leaving at least 50% of the cache available for other purposes.

[0065] Similarly, in a memory system component (such as memory controller 24) with a limited amount of bandwidth available for servicing memory transactions, minimum and / or maximum bandwidth thresholds can be specified for each partition identifier. If, within a given time period, a memory transaction specifying a given partition identifier has already used less than the minimum bandwidth, the memory transaction associated with that partition identifier can be prioritized; if the maximum bandwidth or more has already been used for a memory transaction specifying the same partition identifier, a lower priority can be applied to that transaction.

[0066] These control mechanisms will be discussed in more detail below. It should be understood that these are merely two examples of ways to partition memory system resources based on the software execution environment that issues the corresponding transaction. Generally, by allowing different processes to “see” different partitions of the resources provided by the memory system, this allows for limiting performance interactions between processes to help address the problems discussed above.

[0067] Similarly, partition identifiers associated with memory transactions can be used for partition performance monitoring within a memory system. This allows for the tracking of separate sets of performance monitoring data for each partition identifier, enabling the identification of information specific to a given software execution environment (or group of software execution environments). This makes it easier to identify sources of potential performance interactions than when performance monitoring data is recorded for all software execution environments as a whole. This can also aid in diagnosing potential performance interaction effects and in identifying possible solutions.

[0068] The following describes the architecture for setting control partition identifiers, marking memory transactions based on partition identifiers for the corresponding software execution environment, routing partition identifiers through the memory system, and providing partition-based control at memory system components within the memory system. This architecture can be extended to a wide range of uses of partition identifiers. The use of partition identifiers is designed to override them without changing the existing architectural semantics of the memory system, and therefore any required addressing, consistency, and ordering of memory transactions imposed by the specific memory protocols used by the memory system will not be affected by resource / performance monitoring partitioning. When using partition identifiers to control resource allocation, although this may affect the performance achieved when servicing memory transactions for a given software execution environment, it does not affect the results of architecturally efficient computations. That is, partition identifiers do not change the effect or result of a memory transaction (e.g., what data is accessed), but only affect the timing or performance implemented for that memory transaction.

[0069] Figure 4An example of processing element 6 is illustrated in more detail. The processor includes a processing pipeline comprising multiple pipeline stages, including: a fetch stage 40 for fetching instructions from instruction cache 10; a decode stage 42 for decoding the fetched instructions; a dispatch stage 44 including a dispatch queue 46 for queuing instructions while waiting for their operands to become available and issuing instructions for execution when operands become available; an execution stage 48 including multiple execution units 50 for executing different types of instructions to perform corresponding processing operations; and a write-back stage 52 for writing the results of the processing operations to data register 54. Source operands for data processing operations can be read from register 54 by execution stage 48. In this example, execution stage 48 includes an ALU (Arithmetic / Logic Unit) for performing arithmetic or logical operations, a floating-point (FP) unit for performing operations using floating-point values, and a load / store unit for performing load operations that load data from the memory system into register 54 or store data from register 54 into the memory system. It should be understood that these are merely some examples of possible execution units, and other types may be provided. Similarly, other examples may have different pipeline-level configurations. For example, in an out-of-order processor, an additional register renaming level could be provided to remap the architecture register qualifiers specified by instructions to the physical register qualifiers that identify register 54 provided in the hardware, and a reordering buffer could be provided to track the execution and commit of instructions executed in a different order than that fetched from cache 10. Similarly, it is still possible to provide... Figure 4 Other mechanisms not shown, such as branch prediction functionality.

[0070] Processing element 6 has multiple control registers 60, including, for example: a program counter register 62, which stores a program counter indicating the current execution point of the program being executed; an exception level register 64, which stores an indication of the current exception level at which the processor is executing instructions; a security status register 66, which stores an indication of whether processing element 6 is in a non-safe or safe state; and a memory partitioning and monitoring (MPAM) control register 68 (discussed in more detail below), which controls memory system resource and performance monitoring partitioning. It should be understood that other types of control registers may also be provided.

[0071] Processing element 6 has a memory management unit (MMU) 70 for controlling access to the memory system in response to memory transactions. For example, when a load or store instruction is encountered, the load / store unit issues a corresponding memory transaction specifying a virtual address. This virtual address is provided to the memory management unit (MMU) 70, which translates the virtual address into a physical address using address mapping data stored in a translation lookup buffer (TLB) 72. Each TLB entry identifies not only the mapping data that identifies how the address is translated, but also the associated access permission data that defines whether the processor is allowed to read or write the address in the corresponding page of the address space. In some examples, multiple address translation levels may exist, and therefore multiple TLBs may exist, for example, a first-level TLB that provides a first translation level for mapping the virtual address generated by the load / store unit 50 to an intermediate physical address, and a second-level TLB that provides a second translation level for mapping the intermediate physical address to a physical address used by the memory system to identify the data to be accessed. Mapping data for Level 1 TLBs can be set under the control of the operating system, while mapping data for Level 2 TLBs can be set under the control of the hypervisor, for example, to support virtualization. Although Figure 4 For simplicity, this illustrates that the MMU is accessed in response to a data access load / memory unit trigger. However, the MMU can also be accessed when fetch stage 40 needs to fetch instructions that are not yet stored in instruction cache 10, or when instruction cache 10 initiates instruction prefetching to prefetch instructions into the cache before fetch stage 40 actually needs them. Therefore, the virtual address of the instruction to be executed can be similarly translated into a physical address using MMU 70.

[0072] In addition to TLB 72, the MMU may include other types of caches, such as page walk cache 74, which caches data used to identify the mapping data to be loaded into the TLB during page table walks. The memory system may store page tables containing address mapping data for each page in a specified virtual memory address space. TLB 72 may cache a subset of those page table entries for multiple recently accessed pages. A page table walk is initiated if processing element 6 publishes a memory transaction to a page that does not have corresponding address mapping data stored in TLB 72. This can be relatively slow because it may be necessary to traverse multiple levels of page tables in memory to identify the address mapping entry for the required page. To speed up page table walks, the most recently accessed page table entries can be placed in page walk cache 74. These entries are typically page table entries other than the final-level page table entries, which actually specify the mapping for the required page. These higher-level page table entries typically specify where other page table entries for the corresponding address range can be found in memory. By caching at least some levels of the page tables traversed in previous page table walks in page walk cache 74, page table walks to other addresses that share the same initial portion of a page table walk can be performed faster. Alternatively, instead of caching the page table entries themselves, page walk cache 74 can cache the addresses where those page table entries can be found in memory, thus allowing access to a given page table entry to be faster than having to identify those addresses by first accessing other page table entries in memory.

[0073] Figure 5Examples of different software execution environments that can be executed by processing element 6 are shown. In this example, the architecture supports four different exception levels EL0 to EL3, which are sequentially elevated in privilege levels (such that EL3 has the highest privilege exception level, and EL0 has the lowest privilege exception level). Generally, higher privilege levels have greater privileges than lower privilege levels and therefore can access at least some data that cannot be used at lower privilege levels and / or perform some processing operations that cannot be used at lower privilege levels. Application 80 executes at the lowest privilege level EL0. Many guest operating systems 82 execute at privilege level EL1, where each guest operating system 82 manages one or more applications in application 80 at EL0. Virtual machine monitor (also known as hypervisor or host operating system) 84 executes at exception level EL2 and manages the virtualization of the corresponding guest operating system 82. Transitions from lower exception levels to higher exception levels may be caused by exceptional events (e.g., events that require processing by the hypervisor may cause a transition to EL2), while transitions back to lower levels may be caused by returning from processing exceptional events. Some types of exceptional events can be served at the same exception level as the level at which they are acquired, while others may trigger transitions to higher exceptional states. The current exception level register 64 indicates which exception level (EL0 to EL3) the processing element 6 is currently executing code at.

[0074] In this example, the system also supports a partition between a secure domain 90 and a normal (less secure) domain 92. Sensitivity data or instructions can be protected by assigning them to memory addresses marked as accessible only by the secure domain 90, where the processor has hardware mechanisms to ensure that a process executing in the less secure domain 92 cannot access that data or instructions. For example, access permissions set in the MMU 70 can control the partition between secure and non-secure domains, or alternatively, a completely separate secure memory management unit can be used to control the security state partitioning, where separate secure and non-secure MMUs 70 are provided for sub-control within the respective secure states. The transition between the secure domain 90 and the normal domain 92 can be managed by a security monitor process 94 executing at the highest privilege level EL3. This allows for tight control over the transition between domains to prevent non-secure operations 80 or the operating system 82 from accessing data from the secure domain, for example. In other examples, hardware techniques can be used to perform separation between secure state and polling transitions, allowing code in the normal domain 92 to be directly transferred to code in the secure domain 90 without transitioning via a separate security monitor process 94. However, for ease of explanation, the following description will refer to an example that does indeed use a security monitor process 94 at EL3. Within security domain 90, the secure world operating system 96 executes at exception level EL1, and one or more trusted applications 98 can execute at exception level EL0 under the control of the operating system 96. In this example, there is no exception level EL2 in security domain 90 because virtualization is not supported in this security domain, but it can still be provided if needed. An example of an architecture used to support such a security domain 90 could be ARM from Cambridge, UK. ® The Trustzone architecture is provided by Limited. However, it should be understood that other technologies can also be used. Some examples may have more than two security states, providing three or more states with different levels of security associated with them. The security state register 66 indicates whether the current domain is secure domain 90 or non-secure domain 92, and this instructs the MMU 70 or other control units on which access permissions are used to control whether certain data can be accessed or permitted operations.

[0075] Figure 5 Multiple different software execution environments 80, 82, 84, 94, 96, and 98 that can execute on this system are shown. Each of these software execution environments can be assigned a given partition identifier (partition ID or PARTID), or a group of two or more software execution environments can be assigned a common partition ID. In some cases, a separate part of a single process (e.g., a different function or subroutine) can be considered a separate execution environment and assigned a separate partition ID. For example, Figure 6An example is shown where virtual machine VM 3 and the two applications 3741 and 3974 running under it are all assigned PARTID 1, a specific process 3974 running under a second virtual machine VM 7 is assigned PARTID 2, and VM 7 itself and another process 1473 running under it are assigned PARTID 0. Custom partition IDs do not need to be assigned to each software execution environment. A default partition ID can be specified for software execution environments that are not assigned partition IDs. Control over which portions of the partition ID space are assigned to each software execution environment is exercised by software with a higher privilege level, such as a hypervisor running in EL2 controlling the allocation of partitions to the virtual machine operating system running in EL1. However, in some cases, the hypervisor may allow the operating system with a lower privilege level to set its own partition IDs for portions of its own code or for applications running under it. Furthermore, in some examples, secure world 90 may have a partition ID space completely separate from normal world 92, controlled by the secure world OS or the monitoring program EL3.

[0076] Figure 7 An example of MPAM control register 68 is shown. MPAM control register 68 includes multiple partition ID registers 100 (also referred to as MPAM system registers), each corresponding to a respective operating state of the processing circuitry. In this example, partition ID registers 100 include registers MPAM0_EL1 to MPAM3_EL3 corresponding to the respective exception levels EL0 to EL3 in non-security domain 92, and an optional additional partition ID register MPAM1_EL1_S corresponding to exception level EL1 in security domain 90. In this example, no partition ID register is provided for EL0 in the security domain because it is assumed that trusted applications 98 in the security domain are tightly bound to the secure world operating system 96 running those applications, and therefore they can be identified using the same partition ID. However, in other specific implementations, separate partition ID registers may be provided for EL0 in the secure world. Each partition ID register 100 includes fields for up to three partition IDs, as shown in Table 1 below:

[0077] Table 1 :

[0078]

[0079] Table 2 below summarizes which partition ID register 100 is used for memory transactions performed in each operating state, and from which operating states each partition ID register 100 is controlled (i.e., which operating state can update the information specified by the register):

[0080] Table 2 :

[0081]

[0082] The naming convention for the partition ID register, MPAMx_ELy, indicates that the partition ID specified in the partition ID register MPAMx_ELy is used for memory transactions issued by circuit 6 while it is in operating state ELx, and state ELy is the lowest exception level that can access the partition ID register MPAMx_ELy. However, MPAM0_EL1 can be overridden when the current exception level in the non-security domain is EL0—the partition ID in MPAM1_EL1 is used when executing in NS_EL0 when the configuration value PLK_EL0 set in MPAM1_EL1 is set to 1. Therefore, control over EL1 can override control over EL0 when needed. This can be used to constrain all applications running under a particular virtual machine to use the same partition ID to avoid needing to update MPAM0_EL1 every time there is a context switch between applications within the same virtual machine. Although the configuration parameter PLK_EL0 is described in this example as being stored in MPAM1_EL1 (corresponding to the partition ID register with the higher exception level for which the configuration parameter is set), it could also be stored in another control register.

[0083] Typically, when switching between different processes executing in the same state (e.g., different applications in EL0 or different guest operating systems in EL1), an exception event triggers a switch to a higher exception state, where the process running in that state (e.g., an operating system in EL1 or a hypervisor in EL2) subsequently updates the partition ID in the relevant partition ID register 100, and then returns processing to the lower exception state to allow the new process to continue. Therefore, the partition ID associated with a given process can be effectively considered as part of the context information associated with that process, which is saved and restored as part of the processor architecture state when switching from or to that process.

[0084] However, by providing multiple partition ID registers 100 corresponding to different operating states of the system, it is not necessary to update the contents of a single partition ID register every time the operating state changes except during context switching, such as when the operating system (OS) temporarily traps into the hypervisor to perform some action and then returns to the same OS. Such trapping into the hypervisor can be quite common in virtualized systems, for example, if the hypervisor must intervene to provide the OS with a different view of physical resources than actually provided in the hardware. Therefore, by providing multiple partition ID registers 100, using partition IDs to mark memory system transactions will automatically follow changes in exception levels or safe / unsafe states, resulting in faster performance because the partition IDs do not need to be updated every time a change in exception level or safe state occurs.

[0085] Furthermore, for security reasons, providing separate secure and less secure partition ID registers may be preferred, for example, by preventing less secure processes from inferring information about the secure domain from the partition ID used. However, storing partition ID registers by security state is optional, and other implementations may provide only a single version of a given partition ID register shared between the secure and less secure domains (e.g., MPAM1_EL1 could be used, where MPAM1_EL1_S is omitted). In this case, the monitor code executing at EL3 can perform context switching on the information in the partition ID register when switching between the secure and less secure domains.

[0086] Furthermore, generally, control information (such as the partition ID and any associated configuration information specified in the partition ID register 100 associated with a given operating state) is set in response to instructions executed at an exception level higher than the exception level associated with the partition ID register 100. However, this general premise can be overridden for some registers where higher exception level code can set configuration parameters EL1_WRINH, EL2_WRINH, or EL1_S_WRINH, which control whether code executed in a given operating state can set its own partition ID in the corresponding partition ID register. That is, the WRINH configuration value specifies whether a given execution environment is allowed to set the partition ID assigned to itself. While the example below shows the WRINH flag controlling the setting of the partition ID by a given exception level stored together with the partition ID of that exception level in the partition ID register 100 associated with the next highest exception level, it should be understood that these flags can also be stored in separate control registers.

[0087] More specifically, Table 3 lists the information included in each partition ID register 100, and Table 4 summarizes the states from which each partition ID register 100 can be read from or written to. Some registers in register 100 include information specific to that register, as shown in the figure.

[0088] Table 3 :

[0089]

[0090] Table 4 :

[0091]

[0092] The asterisk indicates:

[0093] • When EL1_WRINH=0 in MPAM2_EL2, MPAM1_EL1 can be written from NS_EL1, but when EL1_WRINH=1, MPAM1_EL1 is written from NS_EL1 and then falls into EL2;

[0094] • When EL2_WRINH=0 in MPAM3_EL3, MPAM2_EL2 can be written from EL2, but when EL2_WRINH=0, MPAM2_EL2 is written from EL2 and then trapped in EL3;

[0095] • When EL1_S_WRINH=0 in MPAM3_EL3, MPAM1_EL1_S can be written from S_EL1, but when EL1_S_WRINH=1, MPAM1_EL1_S is written from S_EL1 and then falls into EL3.

[0096] Therefore, attempting to set the partition ID register 100 within the same abnormal state when a higher abnormal state is not permitted will result in an abnormal event, which will trigger a switch to the higher abnormal state. The exception handler in the higher abnormal state can then decide how the partition ID should be set.

[0097] Please note that in the above alternative implementation where MPAM_EL1_S is omitted, MPAM1_EL1 will be R(W*) accessible from both NS_EL1 and S_EL1 (where EL1_WRINH controls whether write access is possible from EL1), and the EL1_S_WRINH configuration parameter can be omitted from register MPAM3_EL3.

[0098] Typically, when processing circuit 6 generates a memory transaction, one of the partition ID registers in partition ID register 100 is selected based on the current operating state as specified above. If the memory transaction is for an access instruction, the transaction is marked with the partition ID derived from the PARTID_I field of the selected partition ID register. A page table walk memory transaction triggered by a missing TLB 72 for an instruction access will use the same partition ID as the instruction access. If the memory transaction is for accessing data, the transaction is marked with the partition ID derived from the PARTID_D field of the selected partition ID register 100 (and similarly, any page table walk access triggered by the MMU after a data access will use the same partition ID as the data access itself). Note that regardless of whether the MMU issuing the page table walk access itself supports resource / performance monitoring partitioning based on partition ID, it can still append the relevant PARTID_D or PARTID_I identifier to the corresponding memory transaction to allow memory system components in another part of the memory system to perform such partitioning. The PARTID_D and PARTID_I fields of a given partition ID register can be set to the same partition ID or different partition IDs.

[0099] Allowing separate partition IDs for data and instruction accesses within the same software execution environment can be useful, enabling different resource control parameters to be used for corresponding instruction and data accesses. An alternative approach is to treat a single partition ID associated with the software execution environment as a whole, but append additional 0s or 1s depending on whether the access is for instructions or data. This would allow memory system components to select different control parameters for instruction and data accesses respectively. However, for a given number of control parameter sets selected based on the partition ID, this approach would imply a 50-50 split of the partition ID space between data and instructions. In implementation, it may often be desirable to have more data partitions than instruction partitions, as it is relatively common for multiple software execution environments to use the same code but execute with different data inputs. Therefore, it could be particularly useful to be able to share a single instruction partition ID across multiple software execution environments while allowing each of these environments to use a different data partition. In this case, appending 0s or 1s to indicate instructions for data access would require defining multiple sets of the same configuration information at the memory system components for each individual case of that common code. In contrast, by providing separate instruction and data partition fields in partition ID register 100, where instruction and data partition IDs are selected from a common ID space, the same partition ID can be reused across different software execution environments, and the partition ID space can be divided between data and instructions as needed, without constraining this division to be a 50 / 50 split. Even though the two partition ID fields in each partition ID register 100 may require some additional storage capacity, this approach also saves resources at the memory system component because fewer sets of control parameters (and therefore less storage) are needed at the memory system component by sharing a single partition across instruction accesses across multiple execution environments.

[0100] Regardless of whether the transaction is for instruction or data access, it is also tagged with a performance monitoring partition ID derived from the PMG field of the selected partition ID register 100. This allows the memory system component to segment performance monitoring, for example, by using the performance monitoring ID of the memory transaction as part of a criterion for determining whether a given performance monitor should be updated in response to a memory transaction. In one implementation, the PMG field can be considered completely independent of the PARTID_D and PARTID_I fields. In this case, the memory system component implementing performance monitoring can determine whether a memory transaction causes an update for a given performance monitor solely based on the performance monitoring partition ID, regardless of the data / instruction partition ID included in the same memory transaction. This provides the advantage that different partitions for instruction / data access can still share the same performance monitoring ID, which will support the collection of combined performance statistics for multiple processes that require different instruction / data access configurations at the memory system component. Thus, by specifying a performance monitoring group ID separate from the partition ID used to control resource allocation at the memory system component, this allows the use of a common set of performance counters to track multiple different software execution environments, even if the resources of these software execution environments are allocated separately.

[0101] Alternatively, another approach is to interpret the PMG field as a suffix to be appended to the corresponding partition ID derived from the PARTID_D or PARTID_I field. Using this method, when a transaction is issued to memory, it is appended with two IDs: one based on the selected PARTID_I or PARTID_D field, and another based on the PMG field, but this PMG field is treated as a characteristic of the instruction / data partition ID rather than its own ID. Therefore, in this case, memory system components can perform resource partitioning based on the first partition ID derived from PARTID_I or PARTID_D, but performance monitoring partitioning based on a combination of the first and second partition IDs derived from the PMG. Using this method, different instruction / data partition IDs can no longer share the same performance monitoring ID, but the advantage is that a shorter PMG field can be used to save hardware costs because the PMG field does not need to differentiate among all possible performance monitoring partitions; it only distinguishes partitions that share the same instruction / data partition ID. For example, this allows the use of 1-bit or 2-bit PMG fields instead of larger fields, which not only saves the cost of control register 68 but also saves wires carrying memory transactions through the memory system. In some implementations, separate PMG suffix fields PMG_D and PMG_I, corresponding to the PARTID_D and PARTID_I fields respectively, can be provided to allow defining separate performance monitoring group attributes for data and instruction access respectively.

[0102] Regardless, the ability to define multiple performance monitoring partitions for each data / instruction partition ID can be useful. On the other hand, it should be understood that other examples could omit the separate performance monitoring ID field entirely, instead using the same partition ID to control both resource management and performance monitoring.

[0103] Figure 8 A device 110 according to some configurations of the present technology is schematically illustrated. Device 110 includes a plurality of processing elements 112 and a plurality of sets of regulation circuits 114. The processing elements 112 and regulation circuits 114 are interconnected to each other via interconnects 116 and connected to a storage hierarchy. In the illustrated configuration, the processing elements 112 include a first processing element 112(A) coupled to a first regulation circuit 114(A), a second processing element 112(B) coupled to a second regulation circuit 114(B), a third processing element 112(C) coupled to a third regulation circuit 114(C), and a fourth processing element 112(D) coupled to a fourth regulation circuit 114(D). It will be apparent to those skilled in the art that the regulation circuits 114 may alternatively be provided as a single regulation circuit, for example, included in the interconnects 116. Each regulation circuit 114 responds to feedback signals received via the interconnects to perform control and modify control to limit bandwidth utilization associated with an identifier (e.g., a partition identifier) ​​assigned to a process operating on the processing element 112.

[0104] In particular, in some configurations, Figure 8 At least one of the cases of the regulating circuit illustrated herein is arranged to perform Figure 9 The illustrated sequence of steps is as follows. The process begins at step S60, where it is determined whether the adjustment circuit has received a transaction feedback signal. If it is determined at step S60 that no feedback signal has been received, the process stops at step S60. If it is determined at step S60 that a transaction feedback signal has been received, the process proceeds to step S62, where resource utilization parameters are determined from the feedback signal. Then, the process proceeds to step S64, where it is determined whether the resource utilization parameters meet resource utilization conditions. If it is determined at step S64 that the resource utilization parameters meet resource utilization conditions, the process proceeds to step S66, where bandwidth utilization available for storage transactions requested by the process associated with the identifier is controlled based on predefined limits assigned to the identifier associated with the process that issued the transaction. If it is determined at step S64 that the resource utilization parameters do not meet resource utilization conditions, the process proceeds to step S68, where the control of bandwidth utilization is modified based on the resource utilization parameters.

[0105] Figure 10The illustration schematically illustrates the storage of information indicating, for example, a process requesting bandwidth utilization by an interconnect, according to some configurations of the present technology. In the illustrated configuration, resource utilization parameters included in the feedback signal provide an indication of the number of processing elements running processes associated with an identifier. When operating in at least one mode, the processing elements can use this information to determine control applied by the regulation circuitry. The information indicating a process requesting bandwidth utilization is stored in a storage table 128, which is updated in response to a transaction request 124 received from one of the processing elements. The transaction request is associated with a source identifier (SourceID) 120 indicating the processing element issuing the transaction request 124 and a partition identifier (PARTID) 124 as an identifier associated with the process issuing the transaction request 124. The storage table 128 is indexed based on the partition identifier 124 using a hash circuit 126. The hash circuit 122 receives at least a portion of the partition identifier 124 and generates a hash value identifying a row within the storage table 128. The identified row 130 is read from storage table 128, and the tag comparison circuit 132 compares the partition identifier 122 with the complete partition identifier stored in the tag field of storage table 128. If the tag comparison circuit 132 determines that the partition identifier 122 matches the tag field of the identified row 130, the remaining fields of the identified row 130 are passed to the SourceID comparison circuit 134. In the illustrated configuration, storage table 128 includes columns for each processing element connected to the interconnect, where the values ​​indicate whether the processing element corresponding to that column has been identified as issuing a transaction associated with the partition identifier stored in the tag field of that row. In the illustrated example, it can be seen that the processing elements corresponding to SourceID S10 and SourceID S11 have each been identified as issuing a transaction associated with partition identifier 0x11. The SourceID comparison circuit 134 determines whether the SourceID 120 associated with the transaction request has been indicated in storage table 128 as having issued one or more transaction requests associated with PARTID 122. If so, the SourceID comparison circuit 134 sends a feedback signal to the processing element with SourceID 120, which indicates the number of processing elements running the process associated with the identifier, i.e., the indication of the source identifier set in columns S00 to S11. If SourceID 120 has not yet been set in the stored identifier entry 130 of the storage table 128, the SourceID comparison circuit 134 sets the source identifier and stores the modified entry in the storage table 128.

[0106] If the tag comparison circuit 132 identifies a mismatch between the tag and the partition identifier 122, the tag comparison circuit can trigger the allocation of a new entry with the partition identifier as the tag based on an allocation strategy. It will be apparent that while an indexed storage structure is used in the illustrated configuration, a set-associative or fully associative storage structure can alternatively be used to store information indicating the processing element issuing a transaction request with a given processing identifier. In some configurations, the information in storage table 128 is recorded within a window and can be marked as invalid (or otherwise reset) at the beginning of the window.

[0107] Figure 11 This schematically illustrates the response of the regulating circuit to the condition according to... Figure 10 The sequence of steps is illustrated in the diagram and executed in response to a feedback signal from a circuit. The process begins at step S80, where it is determined whether a transaction feedback signal has been received. If it is determined at step S80 that no transaction feedback signal has been received, the process remains at step S80. If it is determined at step S80 that a transaction feedback signal has been received, the process proceeds to step S82, where the regulating circuit determines the number of processing elements issuing transaction requests associated with the identifier. The process then proceeds to step S84, where it is determined whether the number of processing elements is equal to one. If it is determined at step S84 that the number of processing elements is equal to one, the process proceeds to step S86, where bandwidth utilization (e.g., transaction rate) is limited based on a predefined limit assigned to the identifier. If it is determined at step S84 that the number of processing elements is not equal to one, the process proceeds to step S88, where bandwidth utilization is controlled based on the predefined limit assigned to the identifier divided by the number of processing elements determined in step S82.

[0108] Figure 12The interaction between a regulation circuit 140 and a memory system 142 according to some configurations of the present technology is schematically illustrated. The regulation circuit is configured with a storage circuit 148 storing instructions for predefined limits, a switching circuit 146, a transaction limiting circuit 150, and a comparator circuit 144. The regulation circuit 140 receives transaction requests and passes them to the memory system 142 based on the CBUSY signal received by the regulation circuit 140 from the memory system 142. The CBUSY signal is a feedback signal indicating whether the memory system indicates how busy the completer is. In some configurations, the CBUSY signal is a three-bit field indicating whether the completer is highly congested, lightly congested, or not congested. The CBUSY signal is passed to the comparator circuit 144, which determines whether the CBUSY signal is equal to 0b000, i.e., whether the CBUSY signal indicates that the memory system is not congested. When the completer is not congested, the comparator circuit 144 outputs logic 0 (indicating that the resource utilization condition is not met), otherwise it outputs logic 1 (indicating that the resource utilization condition is met). The output of the comparator circuit is passed to the switching circuit 146 to control whether the output of the switching circuit 146 is passed to the memory system without being limited based on a predefined limit, or whether the output of the switching circuit 146 is passed to the transaction limiting circuit 150 to limit bandwidth utilization based on a predefined limit. When the output of the comparator circuit 144 is logic zero, the switching circuit 146 receives transaction requests and outputs them to the memory system, and when the output of the comparator circuit 144 is logic 1, it outputs them to the limiting circuit. The limiting circuit 150, in response to receiving one or more transactions, determines whether the transaction will exceed a predefined limit 148. If it will exceed the limit, the transaction limiting circuit 150 takes one or more steps to limit the transaction (e.g., by suspending the request processing element). If it will not exceed the limit, the transaction limiting circuit 150 passes the transaction request to the memory system 142, which takes steps to satisfy the transaction request.

[0109] Figure 13 This schematically illustrates the response of the regulating circuit to the condition according to... Figure 12The sequence of steps is illustrated in the diagram and executed in response to a feedback signal from a circuit. The process begins at step S100, where it is determined whether a transaction feedback signal (e.g., a CBUSY signal) has been received. If it is determined at step S100 that no transaction feedback signal has been received, the process remains at step S100. If it is determined at step S100 that a transaction feedback signal has been received, the process proceeds to step S102. At step S102, it is determined whether the storage transaction is congested (e.g., whether CBUSY is not equal to 0b000). If it is determined at step S102 that the storage transaction is congested, the process proceeds to step S104, where the regulating circuit controls bandwidth utilization based on a predefined limit assigned to the identifier. If it is determined at step S102 that the storage transaction is not congested (e.g., if CBUSY equals 0b000), the process proceeds to step S106, where the transaction is allowed to continue even if the predefined limit is exceeded.

[0110] Figure 14 A specific implementation of a emulator that can be used is illustrated schematically. While the previously described embodiments implement the invention in terms of means and methods for operating specific processing hardware supporting the technologies involved, an instruction execution environment according to the embodiments described herein can also be provided, which is implemented using a computer program. Such computer programs are generally referred to as emulators, in part because they provide a software-based implementation of a hardware architecture. Types of emulator computer programs include simulators, virtual machines, models, and binary converters, including dynamic binary converters. Typically, the emulator implementation can run on a host processor 730, which optionally runs a host operating system 720, thereby supporting the emulator program 710. In some arrangements, multiple emulation layers may exist between the hardware and the provided instruction execution environment and / or multiple different instruction execution environments provided on the same host processor. Historically, powerful processors were required to provide emulator implementations that execute at a reasonable speed, but such approaches may be reasonable in certain situations, such as when it is desirable to run code native to another processor for compatibility or reuse reasons. For example, the emulator implementation may provide additional functionality to the instruction execution environment that is not supported by the host processor hardware, or provide an instruction execution environment that is typically associated with a different hardware architecture. An overview of the simulation is given in the following literature: “Some Efficient Architecture Simulation Techniques”, Robert Bedichek, Winter 1990 USENIX Conference, pp. 53-63.

[0111] With respect to embodiments previously described with reference to specific hardware constructions or features, equivalent functionality may be provided in simulated embodiments by suitable software constructions or features. For example, specific circuitry may be implemented as computer program logic in simulated embodiments. Similarly, memory hardware such as registers or cache memory may be implemented as software data structures in simulated embodiments. Where one or more of the hardware elements referenced in the previously described embodiments are present in an arrangement on host hardware (e.g., host processor 730), some simulated embodiments may utilize the host hardware where appropriate.

[0112] The simulator program 710 can be stored on a computer-readable storage medium (which may be a non-transitory medium) and provides a program interface (instruction execution environment) to the target code 700 (which may include applications, operating systems, and management programs). This program interface is identical to the interface of the hardware architecture modeled by the simulator program 710. Therefore, the simulator program 710 can be used to execute the program instructions of the target code 700 from within the instruction execution environment, so that the host computer 730 does not actually have the aforementioned... Figures 1 to 13 The hardware features of the described device. For example, simulator code 710 may include request processing element program logic 712 and regulating program logic 714, the request processing element program logic being configured to issue a storage transaction in response to a storage access request from a process associated with an identifier, the process running on the request processing element program logic, and the regulating program logic being configured to control bandwidth utilization available for storage transactions requested by the process associated with the identifier. When operating in at least one mode, the regulating program logic is configured to control bandwidth utilization based on transaction feedback signals issued by program logic other than the request processing element program logic and indicating resource utilization parameters: when the resource utilization parameters meet resource utilization conditions, control is applied to limit bandwidth utilization to a predefined limit assigned to the identifier; and when the resource utilization parameters do not meet resource utilization conditions, modifications are applied to the control based on the resource utilization parameters.

[0113] In general, an apparatus, a method, and a computer program are provided. The apparatus includes a request processing element that issues a storage transaction in response to an access request from a process associated with an identifier. The apparatus also includes regulating circuitry that controls the bandwidth available for storage transactions requested by the process associated with the identifier. The regulating circuitry is configured to control the bandwidth based on a transaction feedback signal issued by circuitry other than the request processing element and indicating resource utilization parameters: when the resource utilization parameters meet resource utilization conditions, control is applied to limit utilization to a predefined limit assigned to the identifier; and when the resource utilization parameters do not meet resource utilization conditions, the control is modified based on the resource utilization parameters.

[0114] In this application, the phrase "configured as..." is used to mean that the elements of the device are configured to perform the defined operations. In this context, "configuration" means the arrangement or manner of interconnection of hardware or software. For example, the device may have dedicated hardware that provides the defined operations, or a processor or other processing device may be programmed to perform the function. "Configured as" does not imply that the elements of the device need to be changed in any way to provide the defined operations.

[0115] In this application, a list of features beginning with the phrase “at least one of” means that any one or more of those features can be provided independently or in combination. For example, “at least one of [A], [B], and [C]” covers any of the following options: A only (without B or C), B only (without A or C), C only (without A or B), a combination of A and B (without C), a combination of A and C (without B), a combination of B and C (without A), or a combination of A, B, and C.

[0116] While exemplary configurations of the invention have been described in detail herein with reference to the accompanying drawings, it should be understood that the invention is not limited to those precise configurations, and various changes, additions, and modifications can be made therein by those skilled in the art without departing from the scope of the invention as defined by the appended claims. For example, features of the dependent claims can be combined with features of the independent claims in various ways without departing from the scope of the invention.

Claims

1. An apparatus, the apparatus comprising: A request processing element configured to issue a storage transaction in response to a storage access request from a process running on the request processing element, the process being associated with an identifier; and A regulating circuit configured to control bandwidth utilization available for storage transactions requested by a process associated with the identifier, wherein, when operating in at least one mode, the regulating circuit is configured to control the bandwidth utilization based on a transaction feedback signal issued by circuitry other than the request processing element and indicating resource utilization parameters: When the resource utilization parameters meet the resource utilization conditions, the application control limits the bandwidth utilization to a predefined limit assigned to the identifier. as well as When the resource utilization parameters do not meet the resource utilization conditions, the control application is modified based on the resource utilization parameters.

2. The apparatus of claim 1, wherein the resource utilization parameter indicates the resource utilization by the process associated with the identifier.

3. The apparatus according to claim 1 or claim 2, wherein the resource utilization parameters include indications of resource utilization other than the request processing element.

4. The apparatus according to any of the preceding claims, wherein the resource utilization parameters include processing element identification information, the processing element identification information indicating a processing element running a process associated with the identifier, and the resource utilization conditions include processing element conditions that are satisfied when the requesting processing element is the only processing element identified as running the process associated with the identifier.

5. The apparatus of claim 4, wherein the processing element identification information indicates the number of processing elements running a process associated with the identifier, and the modification includes limiting the bandwidth utilization to a reduced limit based on the number of processing elements.

6. The apparatus of claim 5, wherein the reduction limit is calculated by dividing the bandwidth utilization limit by the number of processing elements running the process associated with the identifier.

7. The apparatus according to any one of claims 4 to 6, the apparatus comprising interconnecting circuitry configured to store processing element utilization information indicating the number of processing elements issuing a transaction request associated with the identifier, wherein the interconnecting elements are configured to issue the feedback signal indicating the number of processing elements.

8. The apparatus of claim 7, wherein the interconnect is configured to apply an aging mechanism to the processing element to utilize information.

9. The apparatus of claim 8, wherein the aging mechanism includes storing the processing element utilization information on a sliding window.

10. The apparatus according to any of the preceding claims, wherein the resource utilization parameter includes a congestion parameter indicating congestion of a storage request, and the resource utilization condition includes a congestion condition satisfied when the congestion parameter exceeds a congestion threshold.

11. The apparatus of claim 10, wherein the limiting circuit is configured to allow the processing element to issue a request at a rate greater than the predefined limit when the modification is applied.

12. The apparatus of claim 10 or claim 11, wherein the limiting circuit is configured to apply a soft limit to the bandwidth utilization when the modification is applied, the soft limit allowing the bandwidth utilization to exceed the predefined limit.

13. The apparatus according to any one of claims 10 to 12, wherein the utilization parameters are issued by a storage hierarchy structure.

14. The apparatus according to any of the preceding claims, the apparatus comprising one or more software-accessible registers, wherein the predefined limit value is stored in the one or more registers.

15. The apparatus according to any of the preceding claims, wherein the identifier is one of a plurality of identifiers, each identifier being assignable to one or more processes, and the predefined limit is set based on each identifier.

16. The apparatus according to any of the preceding claims, wherein the request processing element is capable of operating in an additional mode in which the adjustment circuit is configured to limit the bandwidth utilization to the predefined limit independently of the transaction feedback signal.

17. The apparatus according to any of the preceding claims, wherein the request processing element is configured to suspend the execution of the process in response to the satisfaction of one or more limits.

18. The apparatus according to any of the preceding claims, wherein the identifier associated with the process is defined in a software configurable register.

19. A method comprising: Using a request processing element, a storage transaction is issued in response to a storage access request from a process running on the request processing element, the process being associated with an identifier; as well as The bandwidth utilization can be controlled for storage transactions requested by a process associated with the identifier, wherein, when operating in at least one mode, the adjustment includes controlling the bandwidth utilization based on a transaction feedback signal issued by circuitry other than the request processing element and indicating resource utilization parameters: When the resource utilization parameters meet the resource utilization conditions, the application control limits the bandwidth utilization to a predefined limit assigned to the identifier. as well as When the resource utilization parameters do not meet the resource utilization conditions, the control application is modified based on the resource utilization parameters.

20. A computer program for controlling a host data processing device to provide an instruction execution environment, the computer program comprising: Request processing element program logic, configured to issue a storage transaction in response to a storage access request from a process running on the request processing element program logic, the process being associated with an identifier; and A regulating program logic configured to control bandwidth utilization available for storage transactions requested by a process associated with the identifier, wherein, when operating in at least one mode, the regulating program logic is configured to control the bandwidth utilization based on a transaction feedback signal issued by program logic other than the request processing element program logic and indicating resource utilization parameters: When the resource utilization parameters meet the resource utilization conditions, the application control limits the bandwidth utilization to a predefined limit assigned to the identifier. as well as When the resource utilization parameters do not meet the resource utilization conditions, the control application is modified based on the resource utilization parameters.