Processor, manufacturing method, method of obtaining data, manufacturing system, and medium

Abstract

The invention relates to a processor, a manufacturing method, a method of obtaining data, a manufacturing system and a medium. A processor and a method of obtaining data for a processor are provided. The processor comprises at least a first core, a second core and a distributed cache. The distributed cache comprises a first cache slice connected to the first core and a second cache slice connected to the second core and the first cache slice. The first cache slice is configured to receive a memory access request from the first core and to forward the memory access request to the second cache slice.

Description

Technical Field

[0001] This disclosure relates to cache systems for processors, particularly multi-core processors. This disclosure may be particularly relevant to multi-core graphics processing units (GPUs). Background Technology

[0002] To perform a task, a processing unit (PU) needs to process data. This data is typically stored in an external memory device, which the PU must access to obtain the required data. However, accessing external memory is slow, often constrained by limited bandwidth, and may require multiple accesses to the same data. Therefore, the need to access data from external memory often degrades PU performance. To address this issue, a cache can be set up for the PU.

[0003] A cache is a memory device located inside the PU (Power Processor) or at least closer to the PU than external memory. Because of its proximity to the PU, the PU can access the cache much faster than it can access external memory. Furthermore, caches are typically composed of static RAM (SRAM), while external memory is typically composed of dynamic RAM (DRAM). Even with equal proximity between each memory and the PU, SRAM can be read and written faster than DRAM. By storing data to be processed in a cache, data can be retrieved more quickly, thus improving PU performance.

[0004] However, including cache within the PU occupies chip space that could otherwise be used for additional processing hardware. Furthermore, SRAM is more expensive than DRAM, and including SRAM in the PU increases its manufacturing cost. To limit the cost of cache (both financially and in terms of silicon area), caches are typically significantly smaller than external memory (both physically and in terms of memory capacity). Therefore, caches can only store a subset of the data stored in external memory.

[0005] A cached PU achieves maximum performance gain when its limited memory capacity is prioritized for storing the data most frequently needed by the PU. This prioritization of memory capacity significantly reduces the number of times the PU accesses external memory. When the PU needs a data element, it first checks the cache to find it. If the cache contains the data, the PU can read it from the cache without needing to access external memory, saving significant time and making more efficient use of memory access bandwidth. If the cache does not contain the data, the PU accesses external memory to obtain it and can cache a copy of the data for future use. In this way, the use of a cache reduces the number of times the PU accesses external memory, thereby improving the PU's performance.

[0006] To overcome the performance limitations caused by the limited memory capacity of caches, a multi-level cache system can be implemented. In this system, the PU (Power Controller) has a cache hierarchy that increases memory size but reduces access speed. When the PU needs a data element, it can search the caches (in order corresponding to its position within the cache hierarchy) to find the data. The smallest and fastest cache can be searched first, and if that cache does not contain the data, the next smallest (and next fastest) cache can be searched. Finally, if no cache contains the data, it is obtained from external memory and cached in one of the caches. Summary of the Invention

[0007] To improve the speed at which a set of tasks can be executed, a multi-core processing unit (PU) can be utilized. The cores of the PU can operate in parallel to execute tasks. It is desirable to provide a cache system for each core to further improve the performance of the PU.

[0008] However, setting up a caching system for each core of a GPU can lead to inefficient bandwidth usage. For example, consider a scenario where two or more cores need data elements. When one core needs a data element first, it accesses external memory and copies the data element to its cache. Later, when another core needs the same data element, it also accesses external memory and copies the data element to its cache. In other words, whenever a new core needs the same data element, it must access external memory and copy the data to its cache. This duplication of accessing external memory and copying data elements wastes bandwidth and processing time, thus degrading the performance of a multi-core GPU. Similarly, overall, a portion of the GPU's memory capacity is wasted due to data duplication between the caches of individual cores.

[0009] Furthermore, the lack of a consistent view of memory across cores can lead to processing errors. An example of this is the processing of atomic operations. When an atomic operation involving a data element is performed, the memory address of that data element must not be accessed for any other task. As part of the atomic operation, the core performing the operation locks access to the memory address containing the data element until the atomic operation is complete. In a single-core system, tasks either execute sequentially, in which case there are no conflicts accessing data elements, or the core is multithreaded and tasks execute in parallel. However, because the threads of a core all access the same cache, they maintain a consistent view of memory. When two threads attempt to perform an atomic operation on the same data element, they both attempt to lock access to the data element in the cache. Whichever atomic operation begins first will also lock access to the data element in the cache first, thus forcing the second atomic operation to execute after the first one completes. This respects the order of atomic operations and ensures that atomic operations are not performed on stale data. However, in a multi-core PU where each core has its own cache, the individual cores do not maintain a consistent view of memory at the cache level. When a data element is copied to both the caches of the first and second cores, and the first core performs an atomic operation on the data element, the first core will lock access to the data element in its cache. However, since each core cannot access the caches of any other core, the first core locking access to the data element in its own cache does not prevent the second core from accessing its cached copy of the same data element and performing the atomic operation. This means that atomic operations can be performed out of order and / or on stale data, which can lead to errors. For example, if the first and second atomic operations are scheduled to be performed sequentially on the same data element, and the first atomic operation updates the data element, the second atomic operation will only produce correct output when it processes the updated data element. However, if the first atomic operation is performed by the first core and the second atomic operation by the second core, and each core has its own cache, the second atomic operation can begin using stale data cached in the second core's cache before the first atomic operation has completed. This not only leads to incorrect output of the second atomic operation, but also may cause subsequent operations to produce incorrect results if the output of the second atomic operation is written to memory (e.g., if the second atomic operation is completed after the first atomic operation and rewrites the updated data in memory).

[0010] Even when data elements are not (yet) stored in the cache of a multi-core PU, atomic operations can still pose a performance challenge. When the first core begins an atomic operation, it accesses external memory to cache the relevant data into its cache and locks access to that data in external memory. If, at this time, the second core begins a second atomic operation requiring the same data, the second core also accesses external memory to obtain the data, but finds that the memory address of the data is locked. The second core must wait for the memory address to be unlocked, which means waiting for the first core to complete its atomic operation and write the new data back to external memory. Since the second core cannot know in advance when this operation will occur, it must repeatedly access external memory to check the status of the memory address. Due to the long access time to external memory, this can significantly delay the second core from starting its task and make bandwidth inefficient.

[0011] The goal is to provide a more efficient bandwidth system and improve core consistency, thereby improving performance.

[0012] This summary is provided to introduce, in a simplified form, a series of concepts further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0013] A processor and a method for obtaining data for the processor are provided. The processor includes at least a first core, a second core, and a distributed cache. The distributed cache includes a first cache slice connected to the first core and a second cache slice connected to the second core and the first cache slice. The first cache slice is configured to receive a memory access request from the first core and forward the memory access request to the second cache slice.

[0014] According to one aspect, a processor is provided, comprising: Multiple cores, including a first core and a second core; A distributed cache, comprising multiple cache slices, including a first cache slice and a second cache slice; and The first interconnect between the first cache slice and the second cache slice, The distributed cache is configured to store copies of data at multiple memory addresses in the cache memory. The first cache slice is connected to the first core, and the second cache slice is connected to the second core. The first cache slice is configured to cache a copy of the data stored at the first set of memory addresses among the plurality of memory addresses. The second cache slice is configured to cache copies of data stored at a second set of different memory addresses among the plurality of memory addresses. The first cache slice is configured as follows: Receive a first memory access request from the first core, specifying a target memory address of the memory, wherein the plurality of memory addresses includes the first target memory address; Identify a target cache slice in the first cache slice and the second cache slice based on the target memory address, wherein the target cache slice is a cache slice configured to cache a copy of the data stored at the target memory address; and In response to the target cache slice being identified as the second cache slice, the first memory access request is forwarded to the target cache slice, and The first interconnect is configured to transmit the first memory access request to the second cache slice.

[0015] In some examples, the processor may be a graphics processing unit (GPU), a central processing unit (CPU), or a digital signal processor (DSP).

[0016] The first cache slice may be referred to as being associated with the first core, and the second cache slice may be referred to as being associated with the second core. More generally, there may be a 1:1 association between cache slices of a distributed cache and processor cores. The first cache slice may be directly connected to the first core, and the second cache slice may be directly connected to the second core. More generally, each of the plurality of cache slices may be associated with and (optionally directly) connected to a different core of the processor.

[0017] In some examples, the distributed cache may include more than two cache slices.

[0018] In some examples, the first set of memory addresses can be entirely different from the second set of memory addresses. In other words, the first set of memory addresses and the second set of memory addresses can be a disjoint set.

[0019] The target cache slice can be either the first cache slice or the second cache slice.

[0020] Similar to the first cache slice, the second cache slice can be configured to receive a second memory access request from the second core, wherein the second memory access request identifies a second target memory address, and wherein the second target memory address is part of one of a first set of memory addresses and a second set of memory addresses. The second cache slice can be configured to identify the second target cache slice and forward the second memory access request to the second target cache slice in the same manner as the first crossbar switch is configured to identify the first target cache slice and forward the first memory access request to the first target cache slice. The second target cache slice can be the same cache slice as the first target cache slice. In some examples, the second target cache slice can be the first cache slice.

[0021] The memory can be an external memory device, such as DRAM.

[0022] The first cache slice may include: a first cache library configured to cache a copy of the data stored at the first set of memory addresses; and a first crossbar switch connected to the first cache library. The second cache slice may include: a second cache library configured to cache a copy of the data stored at the second set of memory addresses; and a second crossbar switch connected to the second cache library. The first crossbar switch may be configured to: receive the first memory access request from the first core; identify a target cache library among the first and second cache libraries based on the target memory address, wherein the target cache library is a cache library configured to cache a copy of the data stored at the target memory address; and forward the first memory access request to the target cache library. The first interconnect may be configured to transmit the first memory access request to the second crossbar switch when the target cache library is identified as the second cache library.

[0023] The first core can be connected to the first crossbar switch, and the second core can be connected to the second crossbar switch. The crossbar switch for each cache slice can be configured to receive memory access requests from the cores connected to that cache slice.

[0024] The first crossbar switch and the second crossbar switch can be directly connected via the first interconnect, or they can be indirectly connected via additional crossbar switches in other cache slices and by additional interconnects. For example, the first crossbar switch can be connected to a third crossbar switch in a third cache slice by an interconnect, and the third crossbar switch can be connected to the second crossbar switch by another interconnect. In this way, the first crossbar switch and the second crossbar switch can be indirectly connected, and the first interconnect can transmit memory access requests to the third cache slice via the second cache slice. In other words, the first interconnect can transmit memory access requests to the cache slice directly or indirectly. More generally, each crossbar switch in the distributed cache can be directly connected to at least one other crossbar switch by an interconnect. The interconnect can be unidirectional or bidirectional.

[0025] The cache library of the distributed cache can be physically addressed.

[0026] Each cache library comprises a set of cache addresses. Each cache library can be configured to queue access to each of its cache addresses, such that only one memory access request can be executed for a cache address at any given time. For example, if a cache library receives more than one memory access request specifying the same target memory address, the cache library can serialize the execution of the memory access requests based on the time each memory access request is issued. Alternatively, the crossover switch of a cache slice can be configured to queue access to each cache library within that cache slice.

[0027] Each cache library in a distributed cache can be configured to store copies of data associated with a set of memory addresses, and these sets can be completely disjoint. For example, in a processor that includes a third cache slice comprising a third cache library configured to store copies of data at a third set of memory addresses in the cache memory, the first, second, and third sets of memory addresses can all be different because each memory address can only exist in one of the sets. Therefore, data is not duplicated across the cache libraries of the distributed cache. This means that for a given cache memory capacity, more data can be cached in the distributed cache.

[0028] Similar to the first crossbar switch, the second crossbar switch can be configured to receive a second memory access request from the second core, wherein the second memory access request identifies a second target memory address, and wherein the second target memory address is a portion of either the first set of memory addresses or the second set of memory addresses. The second crossbar switch can be configured to identify a second target cache library and forward the second memory access request to the second target cache library in the same manner as the first crossbar switch is configured to identify the first target cache library and forward the first memory access request to the first target cache library. The second target cache library can be the same cache library as the first target cache library. In some examples, the second target cache library may not reside in the second cache slice. For example, the first cache library can be the second target cache library.

[0029] The first crossbar switch can be configured to transmit the first memory access request to the second crossbar switch via the first interconnect when the target cache library is identified as the second cache library. The second crossbar switch can be configured to: receive the first memory access request via the first interconnect when the target cache library is the second cache library; and send the first memory access request to the second cache library when the target cache library is the second cache library.

[0030] The processor may further include a third core, a third cache slice, and a second interconnect between the second cache slice and the third cache slice. The third cache slice may be connected to the third core. The third cache slice may include: a third cache library configured to cache copies of data stored at a third set of memory addresses in the plurality of memory addresses; and a third crossbar switch connected to the third cache library. The first crossbar switch may be configured to transmit the first memory access request to the second crossbar switch via the first interconnect when the third cache library is identified as the target cache library. The second crossbar switch may be configured to transmit the first memory access request to the third crossbar switch via the second interconnect when the target cache library is identified as the third cache library. The third crossbar switch may be configured to send the first memory access request to the third cache library when the target cache library is identified as the third cache library.

[0031] In other words, the first crossbar switch and the second crossbar switch can route the first memory access request to a cache slice containing the target cache library around the distributed cache.

[0032] It should be understood that when a crossbar switch sends a memory access request to a cache library in the same cache slice as the crossbar switch, the memory access request is sent internally to that cache slice. In other words, the memory access request is not sent outside the cache slice via interconnects.

[0033] Each cache library can be associated with an identifier. The first crossbar switch can be configured to use a hash function to map each memory address in the memory to an identifier of a cache library configured to cache a copy of the data stored at that memory address. The first crossbar switch can be configured to use the hash function to identify the target cache library based on the target memory address.

[0034] Each cache library's identifier can be unique, meaning no two cache libraries have the same identifier. Specifically, in the example where the identifier is unique, each cross switch uses the same hash function. However, in some examples, the identifier may not be unique, and the cross switches may use different hash functions.

[0035] The processor may be partitionable to partition the core into at least a first domain and a second domain. The first domain includes the first core, the second core, the first cache slice, and the second cache slice. The second domain includes the third core and the third cache slice. The first and second crossbar switches may be configured to use a first hash function, and the third crossbar switch may be configured to use a second hash function. The first hash function may be configured such that, for any target memory address, the first crossbar switch can identify either the first or second cache library as the target cache library, but cannot identify the third cache library as the target cache library; and for any target memory address, the second crossbar switch can identify either the first or second cache library as the target cache library, but cannot identify the third cache library as the target cache library. The second hash function may be configured such that, for any target memory address, the third crossbar switch can identify the third cache library as the target cache library, but cannot identify either the first or second cache library as the target cache library.

[0036] In this way, hash functions can facilitate the partitioning of processor cores, ensuring that cores in different domains do not use the same cache library. More specifically, cores in each domain can only access the cache library within that domain.

[0037] Processor partitioning can be a logical partitioning implemented in software. Processor partitioning can be achieved by programming the crossbar switches of a first and second cache slice to use a first hash function, and by programming the crossbar switches of a third cache slice to use a second hash function. Partitioning can be dynamic because the processor can be partitioned during use and can potentially be restored to an unpartitioned state during use.

[0038] In some examples, when a processor is partitioned, domains are used to execute different tasks that require different data. However, in some cases, domains may need the same data. Therefore, in some examples, a first cache library and a third cache library can be configured such that a first set of memory addresses and a third set of memory addresses share at least one memory address. In other words, the first cache library and the third cache library can be configured such that at least some data can be duplicated between the first cache library and the third cache library. More generally, a distributed cache can be configured such that data can be duplicated between different domains, but not within a domain.

[0039] In some examples, the first and second fields may include the same number of cache slices (of the same size). In this case, the identifiers used in the first and second fields can be the same, and the first and second hash functions can be the same. However, if the first and second fields include different numbers of cache slices, the first and second hash functions will be different.

[0040] The first crossbar switch may include multiple output channels, wherein at least a first output channel is connected to a first cache library, and at least a second output channel is connected to a first interconnect. The first crossbar switch may include a routing table, wherein the routing table indicates a predetermined output channel to the cache library associated with each identifier. The first crossbar switch may be configured to use the routing table and, based on the identifier of the target cache library, identify the predetermined output channel to the target cache library; and transmit the first memory access request via the predetermined output channel.

[0041] The first output channel and the first cache library can be directly connected, or indirectly connected via one or more additional hardware components. Similarly, the second output channel and the first interconnect can be directly connected, or indirectly connected via one or more additional hardware components. For example, if the first cache slice and the second cache slice are implemented on different silicon dies, a conversion unit can be implemented between the second output channel and the first interconnect. The conversion unit can be configured to convert data between different protocols, thereby enabling data transfer between cache slices using different protocols. The conversion unit may include one or more of the following: a protocol converter, a link controller, a serializer-deserializer (SerDes), and a clock resynchronizer.

[0042] In some examples where the target cache library does not reside in the first cache slice, the first crossbar switch can transmit the identifier of the target cache library along with the memory access request to the second crossbar switch.

[0043] Each crossbar switch in the distributed cache can include multiple output channels and a routing table. The routing table for each crossbar switch can be pre-programmed with a mapping between each cache library in the distributed cache and the output channels of that crossbar switch. The mapping can be configured, for example, to provide the most direct route to the target cache library (the route with the least hop delay).

[0044] The first output channel of the first crossbar switch can be connected to a first cache library, and the second output channel of the first crossbar switch can be connected to a second crossbar switch via a first interconnect. The first output channel of the second crossbar switch can be connected to a second cache library. If the first cache library is the target cache library, the routing table of the first crossbar switch can be programmed to map identifiers to the first output channel. If the second or third cache library is the target cache library, the routing table of the first crossbar switch can be programmed to map identifiers to the second output channel. More generally, the routing table of the first crossbar switch can be programmed to map identifiers of any cache library not residing in the first cache slice to the second output channel.

[0045] The first and second crossbar switches can be configured to use the same hash function. The first and second crossbar switches can use different routing tables.

[0046] The topology of a distributed cache describes the physical connections between cache slices, or in other words, which cache slices are directly connected by interconnects. The routing table defines which interconnects in the topology are used and how they are used. The mapping between cache library identifiers in the routing table and the output channels of crossbar switches is constrained by the distributed cache topology.

[0047] The target cache library can be configured to search the target cache library to find a cached copy of the data stored at the target memory address when the first memory access request is a read request. The target cache library can be configured to read the data stored at the target memory address from the target cache library in response to finding a cached copy of the data. The target cache library can also be configured to read the data stored at the target memory address from the memory in response to not finding the data.

[0048] The target cache library can also be configured to output the data stored at the target memory address to the first core (or more generally, to the core that issued the memory access request) in response to the search finding a cache copy of the data.

[0049] The target cache library can also be configured to cache a copy of the data stored at the target memory address in response to the search not locating a copy of the data in the target cache library, and provide the data to the requesting core.

[0050] The first core can be configured to write to the target cache library when the first memory access request is a write request.

[0051] Multiple cache slices can be connected using one of the following topologies: linear topology, ring topology, partially cross-linked ring topology, densely cross-linked ring topology, fully connected topology, and hybrid topology.

[0052] In a linear topology, at least two cache slices are each directly connected to exactly one other cache slice, and at least one cache slice may be directly connected to exactly two other cache slices. In a ring topology, each cache slice is directly connected to exactly two other cache slices to define the ring topology. In a partially crosslinked ring topology, each cache slice is directly connected to at least two other cache slices to define the ring topology, at least two cache slices are each directly connected to exactly two other cache slices, and at least two cache slices are each directly connected to at least three other cache slices. In a densely crosslinked ring topology, each cache slice is directly connected to at least three other cache slices, and at least two cache slices are not directly connected to each other. In a fully connected topology, each cache slice is directly connected to every other cache slice. In a hybrid topology, at least one cache slice is directly connected to at least three other cache slices, and at least one cache slice is directly connected to exactly one other cache slice.

[0053] The first core may include a first cache. The first cache may be configured as a copy of data stored in cache memory. The first core may be configured to: search the first cache for data stored at the target memory address; and in response to the search not finding the data in the first cache, transmit the first memory access request to the first cache slice.

[0054] In response to the search not finding the data in the first cache, the first core may transfer the first memory access request to the first cross switch of the first cache slice.

[0055] The first core can access the first cache faster than the first core can access the first cache slice. In other words, the first cache can be a cache at a lower level than the distributed cache. The second core can include a second cache similar to the first cache. More generally, each core in the processing unit can include a cache at a lower level than the distributed cache, and each core can be configured to search the lower-level cache for a copy of the data stored at the target memory address before transferring a memory access request to the distributed cache.

[0056] The first cache may include a compressor and a decompressor. The compressor may be configured to: compress a first set of uncompressed data stored in the first cache; and provide the compressed first set of data to the first cache slice. The decompressor may be configured to: receive a second set of data from the distributed cache, wherein the second set of data is compressed; and decompress the second set of data.

[0057] The decompressed second set of data can be cached in the first cache.

[0058] According to another aspect, a method for obtaining data for a processor is provided. The processor includes: a plurality of cores, including a first core and a second core; a distributed cache, including a plurality of cache slices, including a first cache slice and a second cache slice; and a first interconnect between the first cache slice and the second cache slice. The distributed cache is configured to cache copies of data stored at a plurality of memory addresses. The first cache slice is configured to cache copies of data stored at a first set of memory addresses among the plurality of memory addresses, and the second cache slice is configured to cache copies of data stored at a second set of different memory addresses among the plurality of memory addresses. The method includes: receiving a first memory access request specifying a target memory address of the memory by a first cache slice, wherein the plurality of memory addresses includes the target memory address; identifying a target cache slice in the first cache slice and a second cache slice based on the target memory address, wherein the target cache slice is a cache slice configured to cache a copy of data stored at the target memory address; and forwarding the first memory access request to the target cache slice by the first cache slice in response to the target cache slice being identified as the second cache slice, wherein a first interconnect is configured to transmit the first memory access request to the second cache slice.

[0059] The first cache slice may include: a first cache library configured to cache a copy of the data stored at the first set of memory addresses; and a first crossbar switch connected to the first cache library. The second cache slice may further include: a second cache library configured to cache a copy of the data stored at the second set of memory addresses; and a second crossbar switch connected to the second cache library. The method may include: receiving the first memory access request by the first crossbar switch; identifying a target cache library among the first and second cache libraries based on the target memory address, wherein the target cache library is a cache library configured to cache a copy of the data stored at the target memory address; and forwarding the first memory access request to the target cache library by the first crossbar switch. The first interconnect may be configured to transmit the first memory access request to the second crossbar switch when the target cache library is identified as the second cache library.

[0060] The forwarding may include: when the target cache library is identified as the second cache library: the first crossbar switch uses a routing table to identify the output channel of the first crossbar switch leading to the target cache library; and the first crossbar switch transmits the memory access request to the second crossbar switch via the first interconnect. The method may further include: the second crossbar switch receiving the memory access request; the second crossbar switch identifying the target cache library based on the target memory address; and the second crossbar switch sending the first memory access request to the second cache library.

[0061] The method may further include: when the first memory access request is a read request: searching the target cache library to find a cache copy of the data stored at the target memory address; in response to the search finding the data, reading the data stored at the target memory address from the target cache library; and in response to the search not finding the data, reading the data from the memory at the target memory address.

[0062] The method may further include: receiving a second memory access request specifying the target memory address by the second crossbar switch; identifying the target cache library by the second crossbar switch; and forwarding the second memory access request to the target cache library by the second crossbar switch.

[0063] The method may further include: receiving a first memory access request by the target cache library; receiving a second memory access request by the target cache library, wherein the target cache library receives the first memory access request before receiving the second memory access request; locking access to a cached copy of the data; reading the cached copy of the data by the first core; rewriting at least a portion of the cached copy of the data by the first core with updated data; unlocking access to the cached copy of the data; locking access to the cached copy of the data after unlocking access to the cached copy of the data; reading the cached copy of the data by the second core; rewriting at least a portion of the cached copy of the data by the second core with updated data; and unlocking access to the cached copy of the data.

[0064] Access to a cached copy of data can be locked by a cache slice that stores the cached copy of the data. In some examples, the crossbar switch of the cache slice and / or the cache library that stores the cached copy of the data can lock and unlock access to the cached copy of the data.

[0065] In this way, access to data in the distributed cache library can be serialized. This prevents the kernel from reading stale data from the distributed cache, which could lead to processing errors.

[0066] Each cache library can be associated with an identifier. The steps of identifying a target cache library may include mapping a target memory address to a target cache library using a hash function via a first crossbar switch.

[0067] The processor may further include a third core, a third cache slice, and a second interconnect between the second cache slice and the third cache slice. The third cache slice may be connected to the third core and may include: a third cache library configured to cache copies of data stored at a third set of memory addresses in the plurality of memory addresses; and a third crossbar switch connected to the third cache library. The method may further include: when the target cache library is identified as the third cache library: transmitting a first memory access request to the second crossbar switch via the first interconnect by the first crossbar switch; receiving the first memory access request by the second crossbar switch; transmitting the first memory access request to the third crossbar switch via the second interconnect by the second crossbar switch; receiving the first memory access request by the third crossbar switch; and sending the first memory access request to the third cache library by the third crossbar switch.

[0068] The processor may further include a third core, a third cache slice, and a second interconnect between the second cache slice and the third cache slice. The third cache slice may be connected to the third core and may include: a third cache library configured to cache copies of data stored at a third set of memory addresses in the plurality of memory addresses; and a third crossbar switch connected to the third cache library. The method may further include: partitioning the processor into a first domain and a second domain, the first domain including the first core, the second core, the first cache slice, and the second cache slice, and the second domain including the third core and the third cache slice; configuring the first crossbar switch and the second crossbar switch to use a first hash function; and configuring the third crossbar switch to use a second hash function. The first hash function can be configured such that, for any target memory address, the first crossbar switch can identify either the first cache library or the second cache library as the target cache library, and cannot identify the third cache library as the target cache library; and for any target memory address, the second crossbar switch can identify either the first cache library or the second cache library as the target cache library, and cannot identify the third cache library as the target cache library. The second hash function can be configured such that, for any target memory address, the third crossbar switch can identify the third cache library as the target cache library, and cannot identify either the first cache library or the second cache library as the target cache library.

[0069] The method may further include configuring the routing table of the second cross switch such that the routing table does not recognize the output channel leading to the third cross switch.

[0070] More generally, the routing table for each cross switch within a domain that is directly connected to a cross switch in a different domain can be configured such that the routing table does not recognize any output channels leading to the cross switch in a different domain.

[0071] A processor is also provided, configured to perform the methods outlined above. The processor may be implemented in hardware on an integrated circuit.

[0072] A method for manufacturing the processor as described above using an integrated circuit manufacturing system is also provided.

[0073] A method for manufacturing a processor as described above using an integrated circuit manufacturing system is also provided, the method comprising: processing a computer-readable description of the processor using a layout processing system to generate a circuit layout description of an integrated circuit including the processor; and manufacturing the processor according to the circuit layout description using an integrated circuit manufacturing system.

[0074] A computer-readable code is also provided, configured to cause the methods outlined above to be performed when the code is run. A computer-readable storage medium (optionally non-transitory) is also provided on which computer-readable code is encoded.

[0075] An integrated circuit definition dataset is also provided, which, when processed in an integrated circuit manufacturing system, configures the system to manufacture the processor as outlined above. An optional non-transitory computer-readable storage medium may be provided, on which a computer-readable description of a processor is stored, which, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit comprising the processor as outlined above.

[0076] A computer-readable storage medium (optionally non-transitory) is also provided, having stored thereon a computer-readable description of a processor as outlined above, which, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to perform the following operations: process the computer-readable description of the processor using a layout processing system to generate a circuit layout description of an integrated circuit including the processor; and manufacture the processor using an integrated circuit generation system based on the circuit layout description.

[0077] An integrated circuit manufacturing system is also provided, which is configured to manufacture the processors as described above.

[0078] An integrated circuit manufacturing system is also provided, comprising: a computer-readable storage medium (optionally non-transitory) storing thereon a computer-readable description of a processor as outlined above; a layout processing system configured to process the computer-readable description to generate a circuit layout description of an integrated circuit including the processor; and an integrated circuit generation system configured to manufacture the processor according to the circuit layout description.

[0079] The layout processing system can be configured to determine the location information of the logic components of a circuit derived from an integrated circuit description in order to generate a circuit layout description of an integrated circuit containing a processor.

[0080] As will be apparent to those skilled in the art, the above features can be appropriately combined and can be combined with any aspect of the examples described herein. Attached Figure Description

[0081] The example will now be described in detail with reference to the accompanying drawings, in which: Figure 1 It is based on the block diagram of the processor in the example; Figure 2 This is a block diagram of a distributed cache based on the example; Figures 3A to 3F It is a block diagram depicting the topology of each distributed cache. Figure 4A and Figure 4B It is a flowchart illustrating the method executed by the processor according to the example; Figure 5 It is a flowchart illustrating the method executed by the processor according to the example; Figure 6 A computer system implementing a processor is shown; and Figure 7 An integrated circuit manufacturing system for producing integrated circuits containing processors is shown.

[0082] The accompanying drawings illustrate various examples. Those skilled in the art will understand that the element boundaries (e.g., boxes, groups of boxes, or other shapes) shown in the drawings represent one example of a boundary. In some examples, it may be that one element can be designed as multiple elements, or multiple elements can be designed as one element. Where appropriate, common reference numerals are used throughout the drawings to indicate similar features. Detailed Implementation

[0083] The following description is presented with the aid of examples to enable those skilled in the art to make and use the invention. The invention is not limited to the embodiments described herein, and various modifications to the disclosed embodiments will be apparent to those skilled in the art.

[0084] The implementation scheme will now be described using only examples.

[0085] Figure 1 This is a block diagram of a processor 100 in an example graphics processing unit (GPU). The processor 100 communicates with external memory (not shown). The processor 100 includes multiple cores 110 and a distributed cache 120. The multiple cores 110 include: a first core 111, a second core 112, a third core 113, and a fourth core 114. The first core 111 is connected to a first cache slice 121 of the distributed cache 120. The second core 112 is connected to a second cache slice 122 of the distributed cache 120. The third core 113 is connected to a third cache slice 123 of the distributed cache 120. The fourth core 114 is connected to a fourth cache slice 124 of the distributed cache 120.

[0086] Cache slices 121, 122, 123, and 124 of the distributed cache 120 are configured to store data in external cache memory. Specifically, the first cache slice 121 is configured to store data at a first set of memory addresses in external cache memory, and the second cache slice 122 is configured to store data at a second set of memory addresses in external cache memory. Similarly, the third cache slice 123 is configured to store data at a third set of memory addresses in external cache memory, and the fourth cache slice is configured to store data at a fourth set of memory addresses in external cache memory. The first, second, third, and fourth sets of memory addresses are all different because they do not share any memory addresses.

[0087] The first cache slice 121 is connected to the second cache slice 122 via a first interconnect 131. The second cache slice 122 is connected to the third cache slice 123 via a second interconnect 132. The third cache slice 123 is connected to the fourth cache slice 124 via a third interconnect 133. The fourth cache slice 124 is connected to the first cache slice 121 via a fourth interconnect 134. The interconnects are configured to transfer data between cache slices.

[0088] Each cache slice in the distributed cache is configured to receive memory access requests from the core to which it is connected, and / or indirectly from another core via a crossbar switch. For example, a first cache slice 121 may receive a memory access request from a first core 111, or from a second core 112 via a second cache slice 122 and interconnect 131. When a core needs a data element, it issues a memory access request. The memory access request specifies the memory address in external memory where the data element is stored. This memory address is called the target memory address. Each cache slice is configured to identify, upon receiving a memory access request, which cache slice in the distributed cache 120 is configured to cache the data stored at the target memory address. This cache slice is called the target cache slice. When a cache slice receives a memory access request and identifies itself as the target cache slice, the cache slice searches a cache copy of its data to find the cache of the data stored at the target memory address, and in response to finding a cache copy of the data, forwards the data to the core that issued the memory access request. If the target cache slice is not a cache slice that receives memory access requests from the core, the cache slice is configured to forward memory access requests to the target cache slice via interconnects.

[0089] Figure 2 yes Figure 1 A block diagram of a distributed cache 120. The first cache slice 121 of the distributed cache 120 includes a first cache library 211 and a first crossbar switch 221. The second cache slice 122 includes a second cache library 212 and a second crossbar switch 222. The third cache slice 123 includes a third cache library 213 and a third crossbar switch 223. The fourth cache slice 124 includes a fourth cache library 214 and a fourth crossbar switch 224.

[0090] The data stored in cache libraries 211, 212, 213, and 214 in external memory. As explained above, the data is stored at different memory addresses in the external memory of each of the cache slices 121, 122, 123, and 124 of the distributed cache 120. More specifically, the data is stored at the first set of memory addresses in the external memory of the first cache library 211, the second set of memory addresses in the external memory of the second cache library 212, the third set of memory addresses in the external memory of the third cache library 213, and the fourth set of memory addresses in the external memory of the fourth cache library 214. The cache libraries of the distributed cache 120 are N-way associative.

[0091] The first cross switch 221 is connected to the first cache library 211. Similarly, the second cross switch 222 is connected to the second cache library 212, the third cross switch 223 is connected to the third cache library 213, and the fourth cross switch 224 is connected to the fourth cache library 214.

[0092] Each of the crossbar switches is configured to receive memory access requests from the core to which it is connected, and / or indirectly from another core via the crossbar switch. For example, the first crossbar switch 221 may receive a memory access request from the first core 111, or from the second core 112 via the second crossbar switch 222 and interconnect 131. Each crossbar switch is configured to identify, upon receiving a memory access request, which cache library in the distributed cache 120 is configured to cache the data stored at the target memory address. This cache library is referred to as the target cache library. It should be understood that the target cache library resides in a target cache slice. The crossbar switches are also configured to forward memory access requests to the target cache library. When the target cache library and the crossbar switch receiving the memory access request reside in the same cache slice, this means that the memory access request is sent directly to the target cache library. However, when the target cache library and the crossbar switch do not reside in the same core, this means that the memory access request is forwarded to another cache slice in the distributed cache 120 via the interconnect. If the target cache library contains a cached copy of the data element, the crossbar switch of the cache slice including the target cache library is configured to forward the data element back to the core that issued the memory access request. Data is routed in a manner similar to the memory access request—data is sent directly to the core if the cache slice including the target cache library is directly connected to the core, or sent to the core via an interconnect to another cache slice if the cache slice including the target cache library is not directly connected to the core.

[0093] Interconnects 131, 132, 133, and 134 each consist of two unidirectional interconnects. For example... Figure 2 As shown, the first interconnect 131 includes interconnects 131a and 131b. Interconnect 131a is configured to transmit data from the first cross switch 221 to the second cross switch 222, while interconnect 131b is configured to transmit data from the second cross switch 222 to the first cross switch 221. Similarly, interconnects 132, 133, and 134 each include two unidirectional interconnects.

[0094] Each of the crossbar switches 221, 222, 223, and 224 in the distributed cache 120 includes multiple output channels. Each output channel of the crossbar switch is connected to a cache library within the same cache slice as the crossbar switch or to an interconnect that links the crossbar switch to another crossbar switch in the distributed cache 120. Therefore, the first crossbar switch 221 includes a first output channel connected to the first cache library 211, a second output channel connected to interconnect 131a, and a third output channel connected to interconnect 134b. Crossbar switches in other cache slices are configured accordingly.

[0095] Cache slices 121, 122, 123, and 124 can be connected in various different topologies. Two important considerations when choosing a connection topology for distributed cache 120 are: the amount of chip space occupied by the management interconnects, and the degree of separation between each of the management cache slices. The separation between any two slices is counted in "hops." One hop corresponds to one interconnect that must be traversed to move between any two cache slices. For example (still considering...) Figure 2 The first cache slice 121 and the second cache slice 122 are directly connected by interconnect 131 and separated by one hop. The first cache slice 121 and the third cache slice 123 are separated by an intermediate cache slice (either the second cache slice 122 in a clockwise direction or the fourth cache slice 124 in a counterclockwise direction). Therefore, moving from the first cache slice 121 to the third cache slice 123 requires traversing two interconnects, meaning that the first cache slice 121 and the third cache slice 123 are separated by two hops. The smaller the number of hops, the faster data can be transferred between the two cache slices (all other things being equal). The latency caused by traversing interconnects is called hop latency. To reduce the performance impact of hop latency, it is desirable to reduce the maximum number of hops in the distributed cache. However, reducing the number of hops that separate cache slices comes at the cost of increasing the number of interconnects between cache slices.

[0096] Figure 3A An example of a linear topology is shown. This linear topology includes two "end" slices (in... Figure 3A In the first cache slice 121 and the second cache slice 124, each slice is directly connected to only one other cache slice. The remaining slices are "intermediate" slices (second cache slice 122 and third cache slice 123), and each is directly connected to exactly two other cache slices. The linear topology minimizes the number of interconnects in the distributed cache; however, for caches including... n A distributed cache of slices, which also has n-1 The maximum number of hops.

[0097] Figure 3B A ring topology is shown. Figure 1 and Figure 2 Here's an example of a distributed cache topology (120). In a ring topology, each cache slice is directly connected to exactly two other cache slices. Ring topologies utilize an additional interconnect between "end" slices to evolve into linear topologies. Compared to linear topologies, for caches including... n A distributed cache of slices reduces the maximum number of hops to [number]. .

[0098] Figure 3C , Figure 3D and Figure 3E An example of a cross-linked ring topology is shown. A cross-linked ring topology develops a ring topology using at least one additional interconnect that links two cache slices in the ring topology that are not yet directly connected. This may or may not reduce the total maximum number of hops in the topology, but it reduces the number of hops between at least some of the slices to a linear topology. A fifth cache slice 125 has been introduced in these figures.

[0099] Figure 3C An example of a partially crosslinked ring topology is shown. In addition to the connections found in the ring topology, the partially crosslinked ring topology also includes at least one crosslink between two cache slices that are not yet directly connected. (In...) Figure 3C In this configuration, the second cache slice 122 and the fifth cache slice 125 are crosslinked. This has the effect of reducing the hop count between the second and fifth cache slices from 2 to 1, while the total maximum hop count (e.g., between the second and fourth cache slices) remains 2. ) 。 In an interconnected topology, at least two cache slices are connected to three other cache slices, and at least two cache slices are directly connected to exactly two other cache slices. This topology can only be implemented in a distributed cache that includes four or more cache slices.

[0100] Figure 3D An example of a densely cross-linked topology is shown. This topology utilizes additional cross-links to develop on a partially cross-linked topology. Figure 3D In the example, additional crosslinks have been provided to connect the first cache slice 121 and the fourth cache slice 124, and to connect the first cache slice 121 and the third cache slice 123. In a densely crosslinked topology, each cache slice on the ring is connected to (at least) one other cache slice by crosslinks. However, not all cache slices are directly connected to every other cache slice. In other words, each cache slice is directly connected to at least three other cache slices, and at least two cache slices are not directly connected. Therefore, although several pairs of cache slices are separated by single hops, the total maximum number of hops (e.g., between the second and fourth cache slices) is still 2. ) 。 This topology can only be implemented in a distributed cache that includes five or more cache slices.

[0101] Figure 3EAn example of a fully connected topology is shown. In this topology, each cache slice is directly connected to every other cache slice. Therefore, each pair of cache slices is separated by a single hop, so the total maximum number of hops is 1. 。 This topology has the lowest hop latency of any topology; however, it maximizes the number of interconnects required. For distributed caches comprising more than three cache slices, the ring topology and the fully connected topology are only distinguished from each other. In other words, for distributed caches comprising two or three cache slices, the two topologies are identical.

[0102] Figure 3F An example of a hybrid topology is described. A hybrid topology includes: at least one “end” slice directly connected to only one other cache slice; and at least one “link” slice directly connected to at least three other cache slices. Therefore, although several pairs of cache slices are separated by single hops, the total maximum number of hops (e.g., between a third and fourth cache slice) is 2. Hybrid topologies can only be implemented in distributed caches comprising four or more cache slices.

[0103] Figures 3B to 3D Ring topology and Figure 3F Hybrid topologies offer a trade-off between the competing considerations of reducing hop latency and reducing the silicon area occupied by distributed caches. These topologies can be particularly advantageous in examples involving four or more cache slices (and a corresponding number of cores).

[0104] return Figure 1 In addition to the distributed cache 120, the processor 100 includes several other caches. The first core 111 includes an L0 cache 141, which is connected to L1 cache 151 and L2 cache 161. Similarly, the second core 112 includes an L0 cache 142, which is connected to L1 cache 152 and L2 cache 162. The third core 113 includes an L0 cache 143, which is connected to L1 cache 153 and L2 cache 163, and the fourth core 114 includes an L0 cache 144, which is connected to L1 cache 154 and L2 cache 164. The L0, L1, and L2 caches are each configured to cache data stored in external memory. A core can only access the L0, L1, and L2 caches to which it is connected. Therefore, these caches do not form a consistent view of memory among the cores.

[0105] The L0, L1, and L2 caches, together with the distributed cache 120, form a cache hierarchy. For example, considering the first core 111, the L0 cache 141 is the smallest cache and the fastest cache accessed by the first core 111. In the hierarchy, L0 cache 141 is followed by L1 cache 151, then L2 cache 161, and finally the distributed cache 120. When the first core 111 needs a data element, it searches the caches in the hierarchy sequentially (starting from L0 cache 141) until a cache copy of the data element is found, or until all caches have been searched and no cache copy of the data element is found. In this case, the data element is obtained from external memory.

[0106] L0, L1, and L2 caches are each used to store uncompressed data. Conversely, distributed cache 120 and external memory each store compressed data. To facilitate the transfer of compressed data from distributed cache 120 to L2 cache 161 (and vice versa), the L2 cache includes a compressor 171 and a decompressor 181. Compressor 171 receives uncompressed data as input, compresses the data, and outputs compressed data. Decompressor 181 receives compressed data as input, decompresses the data, and outputs decompressed compressed data.

[0107] For many reasons, it is advantageous to implement a distributed cache 120 "between" the L0-L2 cache and external memory. In some alternative cache system designs (excluding distributed cache 120), the L2 cache is the last cache in the hierarchical structure before external memory. In these systems, the L0-L2 cache stores uncompressed data, while external memory stores compressed data. By implementing distributed cache 120 between the L2 cache and external memory, there is no need to modify the L0-L2 cache. Furthermore, by storing compressed data in distributed cache 120, the footprint of distributed cache 120 can be reduced (in terms of silicon area), or conversely, its effective capacity can be increased for a given footprint. Consider an example where processor 100 utilizes an inclusive cache design (where data stored in the L2 cache is duplicated in distributed cache 120). If, in such an example, distributed cache 120 stores uncompressed data, the memory capacity of distributed cache 120 must be at least equal to the total size of the L2 cache. However, if the distributed cache 120 stores compressed data, its memory capacity can be reduced by a factor of the compression ratio. This corresponds to physically smaller memory elements. Furthermore, to maintain a given data transfer bit rate from the distributed cache 120 to multiple cores 110, the distributed cache 120 requires a set number of interconnects between cache slices (due to the limited bandwidth of a single interconnect). In the case of the distributed cache 120 storing compressed data, the minimum number of interconnects required again decreases proportionally to the compression ratio.

[0108] Figure 4A It shows that the result was obtained by Figure 1 The processor 100 (with) Figure 2 The flowchart below shows the method 400 for executing data processing (using a distributed cache topology). In step 401, the L0-L2 caches are searched sequentially to find the data element. In response to any data element not being found in the L0-L2 caches, in step 402, the first core 111 transmits a memory access request to the first crossbar switch 221. The memory access request specifies the memory address (target memory address) in external memory where the data element is stored. At step 410, the first crossbar switch 221 receives the memory access request.

[0109] In step 420, the first crossbar switch 221 identifies the target cache library based on the target memory address specified by the memory access request. Each cache library in the distributed cache 120 is associated with an identifier. A hash function maps each memory address in external memory to an identifier of the cache library configured to cache the data stored at that memory address. The first crossbar switch 221 uses the 421 hash function to map the target memory address to the identifier of the cache library configured to cache the data stored at the target memory address. In this example, it will be assumed (without loss of generality) that the third cache library 213 is the target cache library.

[0110] At each cross switch in the distributed cache 120, the identifier of each cache unit is associated with the output channel of that cross switch in the routing table. The routing table is programmed to associate the identifier of the cache unit with the output channel that provides a predetermined route to that cache unit. In this example, the routing table is programmed to provide the most direct route from the first cross switch 221 to the cache unit.

[0111] In step 430, the first crossbar switch 221 forwards the memory access request to the target cache library. Forwarding involves using a routing table (and based on the identifier of the target cache library) to identify the output path that provides the most direct route to the target cache library. As explained above, due to the topology of this example ( Figure 2 As shown in the diagram, the first cache slice 121 and the third cache slice 123 are separated by two hops in the clockwise and counterclockwise directions. This means that the second output channel and the third output channel each provide the same direct route to the third cache library 213. In this example, and in these cases, the routing table is programmed to route memory access requests clockwise. Therefore, the first crossbar switch 221 identifies the second output channel. In step 432, the first crossbar switch 221 transmits the memory access request to the second crossbar switch 222 via interconnect 131a.

[0112] Figure 4A A loop comprising steps 410, 420, and 430 is described. This loop corresponds to the second crossbar switch 222 receiving a memory access request from the first crossbar switch 221, identifying the target cache library using a hash function, and forwarding the memory access request to the third crossbar switch 223, as described above for the first crossbar switch 221. This loop is partially repeated again when the third crossbar switch 223 receives a memory access request and identifies the third cache library as the target cache library.

[0113] In step 440, the third crossbar switch 223 sends a memory access request to the third cache library 213, and in step 441, the third cache library receives the memory access request. In step 450, the third cache library 213 is searched to locate the data stored at the target memory address.

[0114] If the third cache library 213 does not contain a cached copy of the data, it reads the data from external memory. In step 480, the third cache library 213 caches a copy of the data. In step 481, the third cache library 213 provides the data to the first core 111.

[0115] However, if search 450 finds data in the third cache library 213, then for all other cores, 451 locks access to the cache copy of the data. Then, the first core 111 reads 460 data from the target cache library. (Go to...) Figure 4B After reading and processing the data, the first core 111 rewrites the cached copy of the data in the third cache library 213. Then, it unlocks access to the cached copy of the (rewritten) data.

[0116] By allowing memory access requests from any of the multiple cores 110 to be forwarded to any cache slice in the distributed cache 120, and because data from each external memory address can only be cached in one of the cache libraries of the distributed cache 120 (and only one instance of the cached data), each core maintains a consistent view of memory at the level of the distributed cache 120. By configuring the processor 100 to perform atomic operations in the distributed cache 120, cores no longer face the risk of simultaneously performing atomic operations on different cache copies of the same data element.

[0117] More generally, the consistent view of data provided by the distributed cache 120 improves the bandwidth efficiency of the processor 100 by reducing the number of times multiple cores 110 need to access external memory. For example, if the first core 111 needs a data element that is not currently cached, it can obtain that data and cache it in the distributed cache 120. The next time any core (e.g., the second core 112) needs the same data element, it can obtain the data from the distributed cache 120, even if a cached copy of the data element is not cached in the second cache slice 122. The distributed cache 120 eliminates the need for the second core 112 to independently access external memory to obtain data elements already cached for the first core 111, thereby reducing bandwidth waste and the time spent accessing external memory.

[0118] In some cases, one core can attempt to perform an atomic operation on a cached copy of a data element in distributed cache 120, while another core has already processed that data in the atomic operation. For example, a second core 112 can transmit a second memory access request to a second cache slice 122, the second memory access request specifying the same target memory address as the first memory access request discussed above. Figure 4B As shown, the second crossbar switch 222 receives a second memory access request 490, identifies a target cache library 491, and forwards the second memory access request 492 to the target cache library. This is consistent with the above description regarding... Figure 4A This was accomplished in the same way. For the sake of brevity, it has been changed from... Figure 4B The steps performed by the third cross switch 223 are omitted; however, it should be understood that the third cross switch 223 will perform the same steps as described above. Figure 4A The steps described above correspond to the steps mentioned above.

[0119] In step 493, the third cache library 213 receives the second memory access request. For simplicity, [the following text is missing]. Figure 4B The search step 450 is omitted because the existence of the data in the third cache library has already been established in this example. The second core 112 must then wait for the third cache library 213 to unlock access to the cached copy of the data before proceeding. Once access to the cached copy of the data has been unlocked, the second core 112 can begin its operation, and the third cache library 213 will again lock access to the cached copy of the data. In step 495, the second core 112 reads the data from the third cache library 213. After the second core 112 has processed the data, it can rewrite the cached copy of the data with the updated data. When the second core 112 has completed its atomic operation, the third cache library 213 unlocks access to the cached copy of the data. Locking and unlocking the cached copy of the data forces the cores to perform atomic operations sequentially, thus preventing the processing of stale data.

[0120] To improve the speed at which tasks can be executed, the kernel can read data elements from distributed cache 120 and cache a copy of that data element in its L2 cache. However, as explained above, distributed cache 120 can be used to store compressed data, while the L2 cache can be used to store uncompressed data. To cache the data read from distributed cache 120 in the L2 cache, the kernel can execute... Figure 5 Method 500. In step 510, decompressor 181 decompresses the data provided to it by first core 111. In step 520, the decompressed data is cached in L2 cache 161.

[0121] After the first core 111 has completed its task, it may have updated the data elements in the L2 cache 161. This means that the cache copy of the data stored in the distributed cache 120 is now outdated and must be updated. To do this, the compressor 172 compresses the updated data elements 530 and then provides the compressed data 540 to the first cache slice 121.

[0122] While caching data in the L0-L2 cache can improve processing speed, it can also lead to new consistency issues. When a core updates a data element, it can rewrite a (now stale) cache copy of that data in distributed cache 120. However, other cores that cache stale data in their L0-L2 caches may not be aware that their cache copies of data are stale. To address this, when a core rewrites a cache copy of a data element in distributed cache 120 with updated data, it can also notify other cores in the processor that the cache copy of that data has been rewritten. The data can be identified by its corresponding address in external memory. Any core using its (now stale) cache copy of data in its L0-L2 cache can then retrieve the data from distributed cache 120. The cores of processor 100 can be interconnected by a set of buses, for example, in a ring topology, and notifications can be sent to the cores via these buses.

[0123] In the example above, each of the multiple cores 110 can access data cached in any cache library of the distributed cache 120, thus providing a completely consistent view of memory. However, the processor 100 can be partitioned, allowing cores (and associated cache slices) to be divided into independent domains. For example, the processor 100 can be partitioned into a first core domain including a first core and a second core, and a second core domain including a third core and a fourth core. For core domains to operate completely independently of each other, the distributed cache 120 must also be partitioned into corresponding first and second cache domains. Figure 2The first cache domain 231 includes a first cache slice 121 and a second cache slice 122, and serves a first core domain. The second cache domain 232 includes a third cache slice 123 and a fourth cache slice 124, and serves a second core domain. In this example, the partitioning of the core and distributed cache 120 is a logical partitioning implemented in software. The partitioning of distributed cache 120 is achieved by configuring crossbar switches in each cache domain to use hash functions that map only to cache libraries within that same cache domain. In the previous non-partitioned example, each cache library in distributed cache 120 was associated with a different identifier in a hash function. Depending on the memory access request input to the hash function, the hash function can map to any cache library in distributed cache 120. However, if processor 100 is partitioned, the crossbar switches (first crossbar switch 221 and second crossbar switch 222) of the first cache domain 231 are instead configured to use hash functions that map only to the first cache library 211 and the second cache library 212. Therefore, memory access requests issued by the first core 111 and the second core 112 cannot be forwarded to the third cache slice 123 or the fourth cache slice 124. Similarly, the cross switches (third cross switch 223 and fourth cross switch 224) of the second cache domain 232 can be configured accordingly. In the most extreme example, the cross switches of each cache slice can use a hash function that maps only to the cache library of that cache slice, such that each cross switch can only identify its own cache library as the target cache library. In other words, the distributed cache 120 can be partitioned so that each cache slice itself acts as a different cache, accessible only by the core to which it is connected. The partitioning of the distributed cache 120 can be dynamic, as it can be implemented or changed as the processor 100 is in use. This can be achieved by reprogramming the cross switches during the operation of the distributed cache 120 to utilize different hash functions.

[0124] In some examples, cache libraries in different cache domains can be associated with the same identifier. That is, the identifier only needs to be unique within each cache domain. For example, the first cache library 211 in the first cache domain 231 and the third cache library 213 in the second cache domain 232 can be associated with the same identifier. Similarly, the second cache library 212 and the fourth cache library 214 can be associated with the same identifier. This can be useful because it allows the use of the same hash function across different cache domains.

[0125] In addition to using hash functions that map only to cache libraries within a given cache domain, the crossbar switch of the distributed cache 120 can also use a routing table configured to prevent the crossbar switch from routing memory access requests outside its respective cache domain. Figure 2 In the example, processor 100 is partitioned into a first cache domain 231 and a second cache domain 232. A first crossbar switch 221 can use a routing table that routes data without using interconnect 134, and a second crossbar switch 222 can use a routing table that routes data without using interconnect 132. Third crossbar switches 223 and fourth crossbar switches 224 can use similar routing tables to prevent data from being routed from the second cache domain 232 to the first cache domain 231. It should be understood that, in this configuration, a domain can only contain cores connected to a set of contiguous cache slices. In other words, cache slices within a cache domain must form an unbroken chain of directly connected cache slices.

[0126] In the non-partitioned example described above, the cache libraries in distributed cache 120 cache data from different sets of memory addresses, ensuring that data does not duplicate between cache libraries. When processor 100 is partitioned, the crossbar switch of distributed cache 120 can be configured to ensure that data does not duplicate within a cache domain, but data can duplicate between cache domains. However, in some cases, domains may operate under different operating systems and / or may be used to cache data from different regions of memory. Therefore, it may not be necessary to ensure that data can duplicate between cache domains.

[0127] The crossbar switch of the distributed cache 120 can be configured to use various routing tables and hash functions. This ability to configure the crossbar switch allows for easy (and reversible) partitioning of the processor 100 without requiring modification to the physical hardware of the distributed cache 120. This configurability can be useful if one or more interconnects, crossbar switches, and / or cache libraries in the distributed cache 120 malfunction (or fail to work properly) due to corruption or manufacturing errors. Figure 1 and Figure 2In the example, the routing table used by the first crossbar switch 221 is configured to identify the output path that provides the most direct route to the target cache library. If the target cache library is the second cache library 212, the routing table of the first crossbar switch 221 will identify the second output path. However, due to a manufacturing error or damage during use, interconnect 131a may not function. In this case, the first crossbar switch 221 can be configured to use a routing table that does not utilize interconnect 113a, instead identifying the third output path as providing the most direct route to the second cache library. While this increases the number of hops separating the first cache slice 121 and the second cache slice 122, thereby reducing processor performance, it means that all slices of the distributed cache 120 are still available to all cores of the processor 100, thus maintaining a consistent view of the memory.

[0128] It should be understood that the scope of this disclosure is not limited to the examples above. Many variations are possible, including but not limited to those described below.

[0129] exist Figure 1 In one example, processor 100 is part of a graphics processing unit (GPU). However, in other examples, processor 100 may be part of a central processing unit (CPU) or a digital signal processor (DSP).

[0130] The above description and Figure 1 and Figure 2 The examples depicted relate to GPUs comprising four cores and a corresponding number of cache slices. However, it should be understood that this disclosure is applicable to GPUs (or CPUs) comprising two or more cores. For example, a GPU may include two cores, six cores, or eight cores.

[0131] exist Figure 1 In the examples, each core is depicted as having a direct connection to one of the cache slices of distributed cache 120. For example, the first core 111 is depicted as having a direct connection to the first cache slice 121. More specifically, each core may be connected to a crossbar switch of its cache slice. However, in some examples, the first core 111 may be indirectly connected to the first cache slice 121 via L2 cache 161. For example, the first crossbar switch 221 may be connected to an interface within L2 cache 161 that itself is connected to compressor 171 and decompressor 181 of L2 cache 161. The remaining cores and cache slices may be configured in the same manner. For example, the second core 112 may be indirectly connected to the second cache slice 122 via L2 cache 162.

[0132] exist Figure 1 and Figure 2In the examples, cache slices are described as N-way associative. However, in some examples, cache slices may instead be fully associative or directly mapped.

[0133] exist Figure 2 In some examples, each cache slice is connected to each of its adjacent cache slices by two unidirectional interconnects, each of which is configured to transfer data in opposite directions. In some examples, each cache slice may be connected to each of its adjacent cache slices by one or more bidirectional interconnects, which are capable of transferring data in either direction. In some examples, a cache slice may be connected to its adjacent cache slices by more than two unidirectional interconnects.

[0134] exist Figure 1 and Figure 2 In the examples, distributed cache 120 is used to store compressed data, while L0-L2 caches are used to store uncompressed data. In some examples, distributed cache 120 may also be used to store uncompressed data. In these examples, distributed cache 120 may include one or more compressors and one or more decompressors. In some examples, each cache slice in distributed cache 120 may include a compressor and a decompressor. A decompressor may be configured to receive compressed data from external storage, decompress the data, and provide the decompressed data to a crossbar switch. When a decompressor resides in a cache slice, it may provide the decompressed data to the crossbar switch of that cache slice, so that the decompressed data can be cached in distributed cache 120. Similarly, a compressor may be configured to receive uncompressed data from distributed cache 120 (and specifically, from a cache library or from a crossbar switch in distributed cache 120). The compressor may compress the data and provide the compressed data to external storage.

[0135] exist Figure 1 In the examples, processor 100 is depicted as having a multi-level cache system including an L0 cache, an L1 cache, an L2 cache, and a distributed cache 120. However, in some examples, processor 100 may be implemented without one or more of the L0-L2 caches. In some examples, the distributed cache 120 may be the only cache implemented in processor 100.

[0136] It should be understood that Figure 4A and Figure 4B The method can be generalized to be applied to any GPU core, cache slice, and cache library.

[0137] In some examples, a distributed cache can be connected to two or more processors, allowing the two or more processors to maintain a consistent view of the memory using the distributed cache. In some examples, the two or more processors may include two CPUs, two GPUs, and / or one CPU and one GPU. In some examples, one cache slice in the distributed cache is connected to a core of a first processor, and another cache slice in the distributed cache is connected to a core of a second processor. In some examples, at least one cache slice in the distributed cache is connected to a core of both a first and a second processor. In some examples, each core of two or more processors may be connected to a cache slice, and each cache slice may be connected to a core of each of the two or more processors.

[0138] exist Figure 2 In the examples, each cache slice is depicted as including one cache library. In some examples, each cache slice may include multiple cache libraries. Each cache library in a cache slice may be connected to a crossbar switch of that slice.

[0139] In some examples, the distributed cache 120 can be physically addressed, and the L0-L2 cache can be virtually addressed. In an exemplary implementation of the L0-L2 cache hierarchy, the processor may include a parameter management unit (PMU). The PMU may allocate portions of physical memory in physically addressed external memory to each core of the processor. The PMU may associate portions of physical memory with virtual addresses pointing to virtual pages of memory in external memory. Each core of the processor may include a memory management unit (MMU) that stores a mapping from virtual addresses to physical addresses. When the PMU allocates a portion of external memory to a core, the mapping between the allocated virtual address of that portion of memory and the physical address of that portion of memory may be stored in the core's MMU.

[0140] When searching the core's L0-L2 caches for a copy of data stored in that portion of external memory, the searches are based on virtual addresses provided by the PMU. If the core's L0-L2 caches do not contain a cached copy of the data stored at that virtual address, a mapping stored in the MMU is used to translate the virtual address into a physical address. This physical address can then be used to retrieve the data from external memory. In this exemplary embodiment, the MMU is located between the L2 cache and external memory to facilitate data retrieval from external memory.

[0141] Because the L0-L2 caches of a core are private (because they cannot be accessed by other cores), and in the sense that no core needs to access memory addresses allocated to another core (which can be ensured through appropriate work partitioning), a core does not need to know the virtual addresses used by other cores, and the core's MMU does not need to store a mapping of virtual addresses used by another core. However, in order to maintain a consistent view of memory at the level of distributed cache 120, each core must be able to access any cache library in distributed cache 120. When a portion of external memory is allocated to a core, the cache libraries in distributed cache 120 configured to cache copies of the data stored at that memory address can be in cache slices of different cores. If distributed cache 120 is virtual-addressable, then when the PMU allocates a portion of external memory to a core, the PMU will have to transfer the allocated virtual address and the mapping of the virtual address to the MMU of the core that allocated the portion of memory, and to the MMU of the core connected to the cache slice configured to cache copies of the data stored at that memory address. The MMU of a core connected to a cache slice needs to map the virtual address to a physical address if the cache slice does not contain a cache copy of the data. This physical address can then be used to retrieve data from external memory. The need to transfer virtual addresses and mappings to multiple cores increases the complexity and latency of the mapping operation, thereby reducing performance.

[0142] However, if the distributed cache 120 is physically addressed, there is no need to transfer virtual addresses and mappings to multiple MMUs. The physically addressed distributed cache 120 can reside behind the MMU (i.e., between the core's MMU and external memory). When the core searches its L0-L2 caches and the distributed cache 120 for cache copies of data elements, the virtual address used to search the L0-L2 cache can be mapped to a physical address by the core's MMU and passed to the distributed cache 120. If the distributed cache 120 does not contain cache copies of data, the physical address provided by the core to the distributed cache 120 can be directly used to retrieve data from external memory. Therefore, even when searching cache slices of different cores for data, the core that has only allocated a portion of the memory needs to store the mapping from virtual addresses to physical addresses.

[0143] In addition, the use of physically addressed distributed cache 120 allows distributed cache 120 to be implemented "on top of the existing L0-L2 cache structure", thereby minimizing any changes to the existing virtual addressing L0-L2 cache structure that may be required.

[0144] Processor 100 may include a memory controller connected to external memory. The memory controller operates using physical memory addresses. By implementing physically addressed distributed cache 120, there is no need for mapping from virtual addresses to physical addresses between cache slices of distributed cache 120 and the memory controller. This allows the distributed cache to be located physically closer to the memory controller.

[0145] exist Figure 2 In the example, each cache library in distributed cache 120 is associated with a unique identifier. That is, no identifier is associated with more than one cache library in distributed cache 120. Therefore, each crossbar in distributed cache 120 uses the same hash function to identify the target cache library. However, in some examples, the identifier may not be unique. In the example, in the hash function of the first crossbar 221, the first cache library 211 is associated with identifier "1", the second cache library 212 is associated with identifier "2", and the third cache library is associated with identifier "3". However, in the hash function of the second crossbar 222, the second cache library 212 is associated with identifier "1", the third cache library 213 is associated with identifier "2", and the first cache library 211 is associated with identifier "3". In this way, the hash functions used by the first crossbar 221 and the second crossbar 222 are different. Although the identifier is unique when only one of the hash functions is considered (i.e., the identifier is unique within each hash function), the identifier is not unique between hash functions. It should be noted that in this example, distributed cache 120 constitutes a single domain (in other words, distributed cache 120 is not partitioned). However, the same principle can be applied within a domain of a partitioned cache. In the example above where the identifier is unique, the crossbar switch can forward the memory access request and the unique identifier of the target cache library to another core. Forwarding the unique identifier means that the crossbar switch receiving the memory access request does not need to perform an identification step, thus improving performance. However, in this example where each crossbar switch associates the same cache library with a different identifier, each crossbar switch does need to identify the target cache library.

[0146] It should be understood that method steps 401, 402, 421, 431-432, and 440-497 are optional. In some examples, some or all of these steps may be omitted.

[0147] exist Figure 4AIn some examples, where clockwise and counterclockwise routes around distributed cache 120 to the target cache library are equally direct, the routing table of the first crossbar switch 221 is programmed to route memory access requests in the clockwise direction. In some examples, the routing table may alternatively be programmed to route memory access requests counterclockwise. In some examples, distributed cache 120 may be configured such that each routing table preferentially routes memory access requests clockwise (or counterclockwise), where clockwise and counterclockwise routes are equally direct. In some examples, the direction may be determined by the index of the core issuing the memory access request. For example, a crossbar switch connected to cores with odd-numbered indices (the first and third cores) may route memory access requests counterclockwise, where each route is equally direct, while a crossbar switch connected to cores with even-numbered indices may route memory access requests clockwise, where each route is equally direct.

[0148] It should be understood that data can be returned from the cache library to the requesting core in the same manner as a memory access request being forwarded to the cache library. Data can be passed from the cache library to the crossbar switch and forwarded from the crossbar switch to the requesting core. For this purpose, the memory access request may include an indication of the index of the requesting core, and the routing table of the crossbar switch may map the core index to an output channel, thereby providing a predetermined route to that core. Each crossbar switch may include an output channel, and each crossbar switch is connected to its core through said output channel.

[0149] exist Figure 4A In the example, the third cache library 213 is the target cache library. However, it should be understood that any cache library in the distributed cache 120 can be the target cache library for memory access requests.

[0150] exist Figure 4A In the example, when the first crossbar switch 221 receives the first memory access request, it uses a hash function to identify the target cache library. Based on this, the first crossbar switch 221 forwards the first memory access request to the second crossbar switch 222. When the second crossbar switch 222 receives the first memory access request, it uses a hash function to identify the target cache library and forwards the first memory access request to the third crossbar switch 223. In some examples, when the first memory access request is received from the first core 111, the first crossbar switch 221 can forward the identifier of the target cache library along with the first memory access request to the second core. This eliminates the need for the second crossbar switch 222 (and any subsequent crossbar switches receiving the first memory access request) to identify the target cache library, which can help improve processing speed (especially for topologies with large hop counts).

[0151] Figure 6A computer system in which the (graphics) processing system described herein can be implemented is shown. The computer system includes a CPU 902, a GPU 904, memory 906, and other devices 914, such as a display 916, a speaker 918, and a camera 919. A processing block 910 (corresponding to processor 100) is implemented on the GPU 904. In other examples, one or more of the depicted components may be omitted from the system, and / or the processing block 910 may be implemented on the CPU 902 or within an NNA 908. Components of the computer system can communicate with each other via a communication bus 920. A storage library 912 (which may correspond to external memory) is implemented as part of memory 906.

[0152] Although Figure 6 A specific implementation of the graphics processing system is shown, but it should be understood that a similar block diagram can be drawn for an artificial intelligence accelerator system—for example, by replacing the CPU 902 or GPU 904 with a neural network accelerator (NNA), or by adding an NNA as an additional unit. In such a case, processing block 910 can be implemented in the NNA.

[0153] Figures 1 to 2 The processor 100 is shown as comprising several functional blocks. This is merely illustrative and is not intended to define a strict division between different logical elements of such an entity. Each functional block may be provided in any suitable manner. It should be understood that intermediate values ​​described herein as being formed by the processor need not be physically generated by the processor at any point, and may only represent logical values ​​that conveniently describe the processing performed by the processor between its inputs and outputs.

[0154] The processor described herein may be embodied in hardware on an integrated circuit. The processor described herein may be configured to perform any of the methods described herein. Generally, any of the functions, methods, techniques, or components described above may be implemented in software, firmware, hardware (e.g., a fixed logic circuit system), or any combination thereof. The terms “module,” “functionality,” “component,” “element,” “cell,” “block,” and “logic” may be used herein to generally denote software, firmware, hardware, or any combination thereof. In the case of a software-specific implementation, a module, functionality, component, element, cell, block, or logic represents program code that, when executed on a processor, performs a specified task. The algorithms and methods described herein may be executed by one or more processors that execute code that causes the processor to perform the algorithm / method. Examples of computer-readable storage media include random access memory (RAM), read-only memory (ROM), optical disk, flash memory, hard disk, and other memory devices that may use magnetic, optical, and other techniques to store instructions or other data and are accessible by a machine.

[0155] As used herein, the terms computer program code and computer-readable instructions refer to any kind of executable code for a processor, including code expressed in machine language, interpreted language, or scripting language. Executable code includes binary code, machine code, bytecode, code defining integrated circuits (e.g., hardware description languages ​​or netlists), and code expressed in programming languages ​​such as C, Java, or OpenCL. Executable code can be, for example, any kind of software, firmware, script, module, or library that, when properly executed, processed, interpreted, compiled, or run in a virtual machine or other software environment, causes the processor of a computer system that supports the executable code to perform tasks specified by said code.

[0156] A processor, computer, or computer system can be any kind of device, machine, or special-purpose circuit, or a set or portion thereof having processing capabilities that enable it to execute instructions. A processor can be any kind of general-purpose or special-purpose processor, such as a CPU, GPU, NNA, system-on-a-chip, state machine, media processor, application-specific integrated circuit (ASIC), programmable logic array, field-programmable gate array (FPGA), etc. A computer or computer system may include one or more processors.

[0157] This invention also intends to cover software defining the configuration of hardware as described herein, such as hardware description language (HDL) software, for designing integrated circuits or for configuring programmable chips to perform desired functions. Specifically, a computer-readable storage medium may be provided on which computer-readable program code is encoded in the form of an integrated circuit definition dataset, which, when processed (i.e., run) in an integrated circuit manufacturing system, configures the system to manufacture a processor configured to perform any of the methods described herein, or a processor including any of the devices described herein. The integrated circuit definition dataset may, for example, be an integrated circuit description.

[0158] Therefore, a method for manufacturing a processor as described herein can be provided at an integrated circuit manufacturing system. Furthermore, an integrated circuit definition dataset can be provided, which, when processed in the integrated circuit manufacturing system, enables the method for manufacturing the processor to be executed.

[0159] Integrated circuit definition datasets can take the form of computer code, such as netlists, code for configuring programmable chips, or hardware description languages ​​suitable for manufacturing at any level within an integrated circuit, including register-transfer level (RTL) code, high-level circuit representations (e.g., Verilog or VHDL), and low-level circuit representations (e.g., OASIS (RTM) and GDSII). Higher-level representations (e.g., RTL) that logically define hardware suitable for manufacturing in integrated circuits can be processed at a computer system configured to generate manufacturing definitions of integrated circuits within the context of a software environment that includes definitions of circuit elements and rules for combining these elements to produce the manufacturing definition of the integrated circuit as defined by the representation. As is typically the case where software executes at a computer system to define a machine, one or more intermediate user steps (e.g., providing commands, variables, etc.) may be required to configure the computer system to generate the manufacturing definition of the integrated circuit, executing code that defines the integrated circuit to generate the manufacturing definition of that integrated circuit.

[0160] Now about Figure 7 Describe an example of processing integrated circuit definition datasets at an integrated circuit manufacturing system in order to configure the system to manufacture processors.

[0161] Figure 7 An example of an integrated circuit (IC) manufacturing system 1002 is illustrated, which is configured to manufacture processors as described in any of the examples herein. Specifically, the IC manufacturing system 1002 includes a layout processing system 1004 and an integrated circuit generation system 1006. The IC manufacturing system 1002 is configured to receive an IC definition dataset (e.g., defining a processor as described in any of the examples herein), process the IC definition dataset, and generate an IC (e.g., containing the processor described in any of the examples herein) based on the IC definition dataset. The processing of the IC definition dataset configures the IC manufacturing system 1002 to manufacture integrated circuits containing processors as described in any of the examples herein.

[0162] The layout processing system 1004 is configured to receive and process an IC definition dataset to determine a circuit layout. Methods for determining a circuit layout based on an IC definition dataset are known in the art and may involve, for example, synthesizing RTL code to determine the gate-level representation of the circuit to be generated, for example, in relation to logic components (e.g., NAND, NOR, AND, OR, MUX, and FLIP-FLOP components). By determining the location information of the logic components, the circuit layout can be determined based on the gate-level representation of the circuit. This can be done automatically or with user intervention to optimize the circuit layout. Once the layout processing system 1004 has determined the circuit layout, it can output the circuit layout definition to the IC generation system 1006. The circuit layout definition may be, for example, a circuit layout description.

[0163] As is known in the art, IC generation system 1006 generates ICs according to a circuit layout definition. For example, IC generation system 1006 may implement a semiconductor device manufacturing process for generating ICs, which may involve a multi-step sequence of photolithography and chemical processing steps during which electronic circuits are gradually formed on a wafer made of semiconductor material. The circuit layout definition may be in the form of a mask, which can be used in the photolithography process to generate ICs according to the circuit definition. Alternatively, the circuit layout definition provided to IC generation system 1006 may be in the form of computer-readable code, which IC generation system 1006 can use to form a suitable mask for generating ICs.

[0164] The various processes performed by the IC manufacturing system 1002 can all be implemented in one location, for example, by one party. Alternatively, the IC manufacturing system 1002 can be a distributed system, allowing some processes to be performed at different locations and by different parties. For example, some of the following stages can be performed at different locations and / or by different parties: (i) synthesizing RTL codes representing an IC definition dataset to form a gate-level representation of the circuit to be generated; (ii) generating a circuit layout based on the gate-level representation; (iii) forming a mask based on the circuit layout; and (iv) using the mask to manufacture the integrated circuit.

[0165] In other examples, processing an integrated circuit definition dataset at an integrated circuit manufacturing system can configure the system to manufacture a processor without processing the IC definition dataset to determine circuit layout. For instance, an integrated circuit definition dataset can define the configuration of a reconfigurable processor, such as an FPGA, and processing the dataset can configure the IC manufacturing system (e.g., by loading the configuration data into the FPGA) to generate a reconfigurable processor with the defined configuration.

[0166] In some implementations, when an integrated circuit manufacturing definition dataset is processed in an integrated circuit manufacturing system, it can enable the integrated circuit manufacturing system to produce devices as described herein. For example, the integrated circuit manufacturing definition dataset referenced above... Figure 7 The described method configures an integrated circuit manufacturing system to produce devices as described in this article.

[0167] In some examples, an integrated circuit definition dataset may include software running on hardware defined at the dataset, or software running in combination with hardware defined at the dataset. Figure 7 In the example shown, the IC generation system can also be configured by the integrated circuit definition dataset to load firmware onto the integrated circuit according to the program code defined at the integrated circuit definition dataset when manufacturing the integrated circuit, or to additionally provide the integrated circuit with program code for use with the integrated circuit.

[0168] Compared to known implementations, the implementations of the concepts set forth in this application in devices, apparatuses, modules, and / or systems (and methods thereof) can lead to performance improvements. Performance improvements may include one or more of increased computational performance, reduced latency, increased throughput, and / or reduced power consumption. During the manufacture of such devices, apparatuses, modules, and systems (e.g., in integrated circuits), trade-offs can be made between performance improvements and physical implementation methods, thereby improving manufacturing methods. For example, a trade-off can be made between performance improvements and layout area, thereby matching the performance of known implementations but using less silicon. This can be accomplished, for example, by reusing functional blocks serially or sharing functional blocks among elements of a device, apparatus, module, and / or system. Conversely, the concepts set forth in this application that lead to improvements in the physical implementation of devices, apparatuses, modules, and systems (e.g., reduced silicon area) can be traded against performance improvements. This can be accomplished, for example, by manufacturing multiple instances of a module within a predefined area budget.

[0169] The applicant has independently disclosed each individual feature described herein, as well as any combination of two or more such features, to the extent that such features or combinations can be implemented based on the general knowledge of those skilled in the art, in accordance with the entire specification, regardless of whether such features or combinations of features solve any problem disclosed herein. In view of the foregoing description, those skilled in the art will understand that various modifications can be made within the scope of this invention.

Claims

1. A processor, comprising: Multiple cores, including a first core and a second core; A distributed cache, comprising multiple cache slices, including a first cache slice and a second cache slice; and The first interconnect between the first cache slice and the second cache slice; The distributed cache is configured as a copy of the data stored at multiple memory addresses of the cache memory; The first cache slice is connected to the first core, and the second cache slice is connected to the second core; The first cache slice is configured to cache a copy of the data stored at the first set of memory addresses in the plurality of memory addresses; The second cache slice is configured to cache copies of data stored at a second set of different memory addresses among the plurality of memory addresses; The first cache slice is configured as follows: The system receives a first memory access request from the first core, specifying a target memory address for the memory, wherein the plurality of memory addresses includes the first target memory address. Based on the target memory address, a target cache slice is identified in the first cache slice and the second cache slice, wherein the target cache slice is a cache slice configured to cache a copy of the data stored at the target memory address, and In response to the target cache slice being identified as the second cache slice, the first memory access request is forwarded to the target cache slice; The first interconnect is configured to transmit the first memory access request to the second cache slice; The first cache slice includes: a first cache library configured to cache copies of the data stored at the first set of memory addresses; and a first crossbar switch connected to the first cache library; The second cache slice includes: a second cache library configured to cache a copy of the data stored at the second set of memory addresses; and a second crossbar switch connected to the second cache library; The first cross switch is configured to: Receive the first memory access request from the first core. Based on the target memory address, a target cache library is identified in the first cache library and the second cache library, wherein the target cache library is a cache library configured to cache a copy of the data stored at the target memory address, and Forward the first memory access request to the target cache library; and The first interconnect is configured to transmit the first memory access request to the second crossbar when the target cache library is identified as the second cache library.

2. The processor of claim 1, wherein the first crossbar switch is configured to transmit the first memory access request to the second crossbar switch via the first interconnect when the target cache library is identified as the second cache library, and wherein the second crossbar switch is configured to: When the target cache library is the second cache library, the first memory access request is received via the first interconnect; and When the target cache library is the second cache library, the first memory access request is sent to the second cache library.

3. The processor according to claim 1, wherein the processor further comprises: Third core; Third cache slice; as well as The second interconnect between the second cache slice and the third cache slice, The third cache slice is connected to the third core; The third cache slice includes: a third cache library configured to cache copies of data stored at a third set of memory addresses in the plurality of memory addresses; and a third crossbar switch connected to the third cache library; The first crossbar switch is configured to transmit the first memory access request to the second crossbar switch via the first interconnect when the third cache library is identified as the target cache library. The second crossbar switch is configured to transmit the first memory access request to the third crossbar switch via the second interconnect when the target cache library is identified as the third cache library; and The third cross switch is configured to send the first memory access request to the third cache library when the target cache library is identified as the third cache library.

4. The processor of claim 3, wherein each cache library is associated with an identifier, wherein the first cross switch is configured to use a hash function, wherein the hash function is configured to map each memory address in the memory to an identifier of a cache library configured to cache a copy of the data stored at that memory address, and wherein the first cross switch is configured to use the hash function to identify the target cache library based on the target memory address.

5. The processor of claim 4, wherein the processor is partitionable to partition the core into at least a first domain and a second domain, the first domain comprising the first core, the second core, the first cache slice, and the second cache slice, the second domain comprising the third core and the third cache slice, wherein the first crossbar switch and the second crossbar switch are configured to use a first hash function, and the third crossbar switch is configured to use a second hash function, wherein the first hash function is configured such that: For any target memory address, the first crossbar switch can identify the first cache library or the second cache library as the target cache library, but cannot identify the third cache library as the target cache library; and For any target memory address, the second crossbar switch can identify the first cache library or the second cache library as the target cache library, but cannot identify the third cache library as the target cache library; The second hash function is configured such that, for any target memory address, the third crossbar switch can identify the third cache library as the target cache library, but cannot identify the first cache library or the second cache library as the target cache library.

6. The processor of claim 4, wherein the first crossbar switch includes a plurality of output channels, wherein at least a first output channel is connected to the first cache library and at least a second output channel is connected to the first interconnect, wherein the first crossbar switch includes a routing table, wherein the routing table indicates a predetermined output channel to the cache library associated with the identifier for each identifier, and wherein the first crossbar switch is configured to: The routing table is used, and a predetermined output channel to the target cache is identified based on the identifier of the target cache; and The first memory access request is transmitted via the predetermined output channel.

7. The processor of claim 1, wherein the plurality of cache slices are connected in one of the following ways: A linear topology in which at least two cache slices are each directly connected to exactly one other cache slice, and optionally at least one cache slice is directly connected to exactly two other cache slices; A ring topology, wherein each cache slice is directly connected to exactly two other cache slices to define the ring topology; A partially crosslinked ring topology, wherein each cache slice is directly connected to at least two other cache slices to define the ring topology, wherein each of the at least two cache slices is directly connected to exactly two other cache slices, and wherein each of the at least two cache slices is directly connected to at least three other cache slices; A densely interconnected ring topology, wherein each cache slice is directly connected to at least three other cache slices, and wherein at least two cache slices are not directly connected to each other; A fully connected topology, in which each cache slice is directly connected to every other cache slice; as well as A hybrid topology in which at least one cache slice is directly connected to at least three other cache slices, and at least one cache slice is directly connected to exactly one other cache slice.

8. The processor of claim 1, wherein the first cache slice is directly connected to the first core, and the second cache slice is directly connected to the second core.

9. A method for manufacturing the processor according to claim 1 using an integrated circuit manufacturing system, the method comprising: When an integrated circuit definition dataset is input into an integrated circuit manufacturing system and processed in the integrated circuit manufacturing system, the integrated circuit manufacturing system is configured to manufacture the processor.

10. A non-transitory computer-readable storage medium storing an integrated circuit definition dataset, wherein when the integrated circuit definition dataset is processed in an integrated circuit manufacturing system, the integrated circuit manufacturing system is configured to manufacture the processor according to claim 1.

11. A non-transitory computer-readable storage medium storing a computer-readable description of a processor according to claim 1, wherein, when processed in an integrated circuit manufacturing system, the computer-readable description causes the integrated circuit manufacturing system to manufacture an integrated circuit implementing the processor.

12. An integrated circuit manufacturing system, comprising: A non-transitory computer-readable storage medium on which a computer-readable description of the processor according to claim 1 is stored; A layout processing system configured to process the computer-readable description to generate a circuit layout description of an integrated circuit implementing the processor; as well as An integrated circuit manufacturing system configured to manufacture the processor according to the circuit layout description.

13. A method for obtaining data for a processor, wherein the processor comprises: Multiple cores, including a first core and a second core; A distributed cache, comprising multiple cache slices, including a first cache slice and a second cache slice; and The first interconnect between the first cache slice and the second cache slice, The distributed cache is configured as a copy of the data stored at multiple memory addresses of the cache memory; The first cache slice is connected to the first core, and the second cache slice is connected to the second core; The first cache slice is configured to cache a copy of the data stored at the first set of memory addresses in the plurality of memory addresses; The second cache slice is configured to cache copies of data stored at a second set of different memory addresses among the plurality of memory addresses; and The method includes: The first cache slice receives a first memory access request specifying the target memory address of the memory, wherein the plurality of memory addresses includes the target memory address; The first cache slice identifies a target cache slice in the first cache slice and the second cache slice based on the target memory address, wherein the target cache slice is a cache slice configured to cache a copy of the data stored at the target memory address; and In response to the target cache slice being identified as the second cache slice, the first cache slice forwards the first memory access request to the target cache slice; The first interconnect is configured to transmit the first memory access request to the second cache slice; The first cache slice includes: a first cache library configured to cache copies of the data stored at the first set of memory addresses; and a first crossbar switch connected to the first cache library; The second cache slice includes: a second cache library configured to cache copies of data stored at the second set of memory addresses; and a second crossbar switch connected to the second cache library; and The method further includes: The first memory access request is received by the first crossbar switch; The first crossbar switch identifies the target cache library in the first cache library and the second cache library based on the target memory address, wherein the target cache library is a cache library configured to cache a copy of the data stored at the target memory address; and The first crossbar switch forwards the first memory access request to the target cache library; The first interconnect is configured to transmit the first memory access request to the second crossbar when the target cache library is identified as the second cache library.

14. The method of claim 13, wherein the forwarding includes when the target cache library is identified as the second cache library: The first cross switch uses a routing table to identify the output channel of the first cross switch leading to the target cache library; as well as The memory access request is transmitted from the first crossbar switch to the second crossbar switch via the first interconnect; The method further includes: The memory access request is received by the second cross switch; The target cache library is identified by the second crossbar switch based on the target memory address; and The first memory access request is sent to the second cache library by the second crossbar switch.

15. The method of claim 13, further comprising, when the first memory access request is a read request: Search the target cache library to find a cache copy of the data stored at the target memory address; In response to the search finding the data, the data stored at the target memory address is read from the target cache library; and In response to the search not finding the data, data is read from the memory from the target memory address.

16. The method of claim 13, further comprising: The second crossbar receives a second memory access request specifying the target memory address; The target cache library is identified by the second crossbar switch; as well as The second crossbar switch forwards the second memory access request to the target cache library.

17. The method of claim 13, wherein each cache library is associated with an identifier, and wherein the identification includes mapping the target memory address to the target cache library by the first cross switch using a hash function.

18. The method of claim 17, wherein the processor further comprises: Third core; Third cache slice; as well as The second interconnect between the second cache slice and the third cache slice, The third cache slice is connected to the third core; The third cache slice includes: a third cache library configured to cache copies of data stored at a third set of memory addresses in the plurality of memory addresses; and a third crossbar switch connected to the third cache library; The method further includes: The processor is partitioned into: Includes the first core, the second core, the first cache slice, and the first domain of the second cache slice; and The second domain includes the third core and the third cache slice; Configure the first crossbar switch and the second crossbar switch to use the first hash function; and Configure the third cross switch to use the second hash function. The first hash function is configured such that: For any target memory address, the first crossbar switch can identify the first cache library or the second cache library as the target cache library, but cannot identify the third cache library as the target cache library; and For any target memory address, the second crossbar switch can identify either the first or the second cache library as the target cache library, but cannot identify the third cache library as the target cache library; and The second hash function is configured such that, for any target memory address, the third crossbar switch can identify the third cache library as the target cache library, but cannot identify the first cache library or the second cache library as the target cache library.

19. The method of claim 13, wherein the processor further comprises: Third core; Third cache slice; as well as The second interconnect between the second cache slice and the third cache slice; The third cache slice is connected to the third core; The third cache slice includes: a third cache library configured to cache copies of data stored at a third set of memory addresses among the plurality of memory addresses; and a third crossbar switch connected to the third cache library. The method further includes: when the target cache library is identified as the third cache library: The first memory access request is transmitted from the first crossbar switch to the second crossbar switch via the first interconnect. The first memory access request is received by the second cross switch; The first memory access request is transmitted from the second cross switch to the third cross switch via the second interconnect. The first memory access request is received by the third cross switch; and The first memory access request is sent to the third cache library by the third cross switch.

20. A non-transitory computer-readable storage medium storing computer-readable code on the non-transitory computer-readable storage medium, the computer-readable code being configured to cause the method of claim 13 to be executed when the computer-readable code is run.

Images