Cache access method, spatial parallel processor and storage medium

Abstract

The application provides a cache access method, a spatial parallel processor and a storage medium. The spatial parallel processor comprises a cache controller, a first-level cache array, a plurality of computing unit arrays and a plurality of second-level cache interfaces. Each computing unit is connected with each first-level cache. The method comprises the following steps: a target computing unit sends a temporary command lacking a positioning index to a target first-level cache, and sends target address information to the cache controller in parallel; the target first-level cache performs a pre-start operation, and the cache controller performs an acquisition operation for the positioning index in parallel; the cache controller generates a narrow command carrying the positioning index, and sends the narrow command to the target first-level cache; and the target first-level cache completes the temporary command according to the narrow command, generates a complete cache access command, and performs a corresponding cache access operation. The technical problems of low access bandwidth and high access delay existing in the conventional multi-core processor architecture and the cache access mechanism thereof are solved.

Description

Technical Field

[0001] This application relates to the field of storage technology, specifically to a cache access method, a spatial parallel processor, and a storage medium. Background Technology

[0002] In traditional multi-core processor architectures, caches are distributed progressively further away from the compute units. Each compute unit has a private L1 cache, which shares the clock domain and local data path with its compute unit and serves only the access requests of that unit. The L2 cache, typically deployed in a shared or fragmented manner, is directly connected to the L1 cache, offering slower access speeds but larger storage capacity. In this traditional architecture, each compute unit accesses the cache independently: only after generating a complete cache access command can it send it to its bound L1 cache, triggering the corresponding data read or write operation.

[0003] It is evident that traditional multi-core processor architectures and their cache access mechanisms suffer from insurmountable performance bottlenecks: at the architecture level, the access bandwidth of a single computing unit is limited by the throughput limit of a single cache interface, resulting in low access bandwidth; at the mechanism level, the complete command generation process and storage access process need to be executed strictly serially, leading to high access latency. Summary of the Invention

[0004] This application provides a cache access method, a spatial parallel processor, and a storage medium to solve the technical problems of low access bandwidth and high access latency in traditional multi-core processor architectures and their cache access mechanisms.

[0005] In a first aspect, embodiments of this application provide a cache access method applied to a spatial parallel processor. The spatial parallel processor includes a cache controller, a level 1 cache array, multiple computing unit arrays, and multiple level 2 cache interfaces. Each computing unit array corresponds one-to-one with each level 2 cache interface. Each computing unit is connected to its corresponding level 2 cache through the level 2 cache interface of its respective computing unit array. Each computing unit is interconnected with each level 1 cache. The method includes: The target computing unit sends a temporary command to the target first-level cache indicating a lack of a location index. At the same time, it sends the target address information to the cache controller in parallel. The location index refers to the identifier of the storage resource corresponding to the target data indicated by the cache access request. In response to the temporary command, the target first-level cache performs a pre-startup operation. At the same time, the cache controller performs a parallel acquisition operation for the location index based on the target address information. The cache controller generates a narrow command carrying the location index and sends the narrow command to the target first-level cache. The bit width of the narrow command is much smaller than the bit width of the temporary command. The target first-level cache completes the temporary command based on the narrow command, generates a complete cache access command, and executes the corresponding cache access operation.

[0006] Furthermore, the temporary command is a temporary read command, the target address information includes a target cache tag, and the cache controller performs a parallel acquisition operation for the location index based on the target address information, including: the cache controller determines the target cache path based on the target cache tag, the target cache path refers to the storage resource in the target first-level cache used to store the target cache line, the target cache line stores the target data; and the number of the target cache path is determined as the location index.

[0007] Further, the cache controller determines the target cache path based on the target cache tag, including: the cache controller determines whether there is a cache path in the target first-level cache with the cache tag being the target cache tag; if it exists, the cache path is determined to be the target cache path; if it does not exist, the cache controller determines whether there is an active entry for the target cache line in the status management entries, the active entry being used to indicate that the target cache line is being backfilled to the specified cache path of the target first-level cache; if it exists, the specified cache path is determined to be the target cache path; if it does not exist, the target cache path is determined from the target first-level cache.

[0008] Further, determining the target cache path from the target first-level cache includes: determining whether there is an idle cache path in the target first-level cache with a read occupancy counter value of zero, wherein the read occupancy counter value is used to represent the number of read requests occupying the target cache path; if it exists, then the idle cache path is determined as the target cache path; if it does not exist, then the read occupancy counter value of each cache path in the target first-level cache is monitored until there is a cache path with a read occupancy counter value of zero, and the corresponding cache path is determined as the target cache path.

[0009] Further, after determining the target cache path from the target first-level cache, the method further includes: the cache controller sending a data backfill request to the target second-level cache; the target second-level cache responding to the data backfill request transmitting the target cache line to the target second-level cache interface; the target second-level cache interface backfilling the target cache line into the target cache path through a data transmission bypass, wherein the data transmission bypass includes a first data path and a second data path, the first data path being the data path between the target first-level cache and a designated computing unit, the second data path being the data path between the designated computing unit and the target second-level cache interface, and the computing unit array to which the designated computing unit belongs being the computing unit array corresponding to the target second-level cache interface.

[0010] Furthermore, the temporary command is a temporary direct write command, the target address information includes a partition index segment, and the cache controller performs a parallel acquisition operation for the positioning index based on the target address information, including: the cache controller determines the target secondary cache interface based on the partition index segment; based on the target cache line corresponding to the target data, it determines a target aggregation buffer from multiple aggregation buffers of the target secondary cache interface, the target aggregation buffer being used to batch write all write data corresponding to the target cache line into the target secondary cache; and the number of the target aggregation buffer is determined as the positioning index.

[0011] Further, determining the target cluster buffer from multiple cluster buffers of the target secondary cache interface based on the target cache line corresponding to the target data includes: determining whether there is an active buffer that matches the target cache line among the multiple cluster buffers of the target secondary cache interface; if there is, determining the active buffer as the target cluster buffer; if there is no active buffer, allocating an idle buffer for the target cache line and determining it as the target cluster buffer.

[0012] Furthermore, the target first-level cache completes the temporary command according to the narrow command to generate a complete cache access command, and executes the corresponding cache access operation, including: the target first-level cache fills the number of the target aggregation buffer into the corresponding field of the temporary direct write command to generate a complete direct write command; the target first-level cache transmits the complete direct write command and the target data to the target aggregation buffer of the target second-level cache interface through data transmission bypass.

[0013] Secondly, embodiments of this application provide a spatial parallel processor for executing the method described in the first aspect of this application.

[0014] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the first aspect of this application.

[0015] As can be seen, in this embodiment, based on a spatial parallel processor architecture where each computing unit and each L1 cache are interconnected and the computing unit can directly connect to the corresponding L2 cache through the L2 cache interface of its computing unit array, the access bandwidth of each computing unit and the parallel processing capability of the processor are improved at the hardware level. Furthermore, by sending a temporary command lacking a location index to the target L1 cache and simultaneously sending the target address information to the cache controller in parallel, the target L1 cache can perform a pre-boot operation in advance, while the cache controller performs the location index acquisition process in parallel, thus parallelizing the pre-boot of the storage array and the acquisition of the location index, reducing the inherent latency of the traditional serial access process. The cache controller generates a narrow command carrying the location index and sends it to the target L1 cache, which completes the temporary command according to the narrow command and performs the corresponding cache access operation. Since the bit width of the narrow command is much smaller than that of the temporary command, there is no need to lay out additional signal lines to support the transmission of the complete command, reducing wiring costs and transmission overhead. Therefore, the solution proposed in this application not only solves the problems of high access latency, large full command bit width, and high path resource consumption under the traditional serial memory access mechanism, and improves the memory access throughput efficiency in high-concurrency scenarios, but also further releases the memory access concurrency potential of the spatial parallel architecture, provides efficient memory access support for multiple computing units to collaboratively process the same task flow, and improves the task processing efficiency of the spatial parallel processor. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a structural block diagram of a conventional multi-core processor architecture provided in an embodiment of this application; Figure 2 This is a structural block diagram of a spatial parallel processor provided in an embodiment of this application; Figure 3 This is a flowchart illustrating a cache access method provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating a data backfilling process provided in an embodiment of this application applied to a spatial parallel processor; Figure 5This is a schematic diagram illustrating a data write-through operation applied to a spatial parallel processor, as provided in an embodiment of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the protection set of the present application.

[0019] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] Please see Figure 1 , Figure 1 This is a structural block diagram of a conventional multi-core processor architecture provided in an embodiment of this application, such as... Figure 1 The traditional multi-core processor architecture shown depicts multiple computing units (Core0, Core1, Core2, ...) bound one-to-one with their private L1 cache, forming independent local memory access units. The L2 cache is deployed in a sharded manner, including shard 0 and shard 1. All L1 and L2 caches are interconnected via an on-chip interconnect bus and uniformly interface with the main memory, forming a complete two-level cache hierarchy. It is understood that in practical applications, more layers of cache structures can be designed based on requirements. This application uses a two-level cache hierarchy as an example for simplification, but it does not constitute the only limitation of this application.

[0022] In the aforementioned processor architecture, the access bandwidth of a single computing unit is limited by the throughput limit of a single cache interface, and the access parallelism cannot increase synchronously with the number of computing units, making it difficult to adapt to the computing power expansion requirements of high-parallel computing scenarios. Furthermore, under this architecture, cache access requests are strictly executed serially: each computing unit can only send a complete cache access command to the bound first-level cache, triggering the corresponding data read or write operation. This serial execution mode lengthens the overall latency of a single access request, and the complete access command needs to be transmitted throughout the cache paths at all levels. The large command bit width and high path bandwidth consumption further limit the memory access throughput efficiency in high-concurrency scenarios.

[0023] To address the aforementioned issues, embodiments of this application provide a cache access method, a spatial parallel processor, and a storage medium.

[0024] The following describes a spatial parallel processor provided by an embodiment of this application.

[0025] Please see Figure 2 , Figure 2 This is a structural block diagram of a spatial parallel processor provided in an embodiment of this application. Figure 2 As shown, the spatial parallel processor 20 includes a cache controller 21, a first-level cache array composed of multiple first-level caches 22, multiple computing unit arrays 23, and multiple second-level cache interfaces 24; wherein, the computing unit arrays 23 and the second-level cache interfaces 24 are configured in a one-to-one correspondence, dividing different lower-level memory access paths in space.

[0026] In terms of hardware connectivity, this architecture adopts a fully cross-connected access topology: each computing unit is interconnected with each L1 cache (not shown in the diagram), allowing any computing unit to directly access the storage resources of any L1 cache, breaking through the bandwidth limitations of the traditional multi-core architecture where computing units and L1 caches are bound together. Simultaneously, all computing units within the same computing unit array share the corresponding L2 cache interface, connecting to the corresponding L2 cache through this interface to complete read / write interactions at the next level of storage. Although not shown in the diagram, it is understandable that the cache controller, as the global control center, also has connections with each L1 cache, each computing unit, and each L2 cache interface, responsible for global cache tag management, storage resource allocation, and command scheduling; furthermore, adjacent computing units within the same computing unit array also have connections for low-latency data interaction within the array.

[0027] In terms of physical layout, this embodiment adopts a layered, surrounding arrangement: the L1 cache array is distributed around the cache controller, multiple compute unit arrays are arranged along the periphery of the L1 cache array and placed on different sides of the spatial parallel processor, and the L2 cache interface is arranged adjacent to the corresponding compute unit array. This layout can minimize the wiring length between the modules inside the processor, reduce transmission latency and hardware wiring overhead, and adapt to the timing requirements of high-concurrency memory access.

[0028] In some embodiments, the spatial parallel processor is specifically implemented as a reconfigurable parallel processor (RPP). A single computing unit can integrate functional modules such as a memory port (MP), a processing element (PE), and an inter-column switchbox (ICSB). The memory port is used to interface with the L1 and L2 caches, executing memory access commands and data interaction; the processing element executes the core computing logic; and the inter-column switchbox enables horizontal data forwarding and path reconstruction between computing units.

[0029] Therefore, compared to traditional multi-core processor architectures, the spatial parallel processor provided in this application embodiment has significant advantages in memory access capabilities and parallel computing support. On one hand, access bandwidth is increased by orders of magnitude. In traditional architectures, the memory access throughput of a single computing unit is limited by the interface bandwidth of its private L1 cache. In this architecture, each computing unit can directly access the storage resources of the entire L1 cache, and the access bandwidth expands synchronously with the number of L1 caches and computing units, breaking the bandwidth bottleneck of a single cache and supporting the massive data access needs in high-concurrency computing scenarios. On the other hand, the flexibility of multi-computing unit collaborative scheduling is greatly improved. In traditional architectures, data in the private cache cannot be directly shared by other computing units, and cross-core data interaction requires transit through the L2 cache. This architecture supports multiple computing units accessing the same L1 cache simultaneously, and also supports a single computing unit scheduling multiple L1 cache resources. It can flexibly adapt to parallel tasks of different granularities, providing a hardware foundation for multiple computing units to collaboratively process the same task flow.

[0030] However, the cache access mechanisms used in traditional multi-core processors are unsuitable for the aforementioned spatial parallel processors. Firstly, traditional cache access mechanisms employ a complete command transmission mode, where a single access command includes a complete address field, control information, and data payload. The large command size necessitates the deployment of numerous signal lines between each computing unit and each L1 cache, consuming significant wiring resources and bandwidth. This drastically increases hardware costs and power consumption, ultimately offsetting the bandwidth gains inherent in the spatial parallel processor architecture itself. Secondly, traditional cache access mechanisms employ a strictly serial execution process, where the complete command generation and storage access processes are completed sequentially. This results in high latency for a single request. In the high-concurrency memory access scenarios of spatial parallel processors, this serial latency accumulates with the number of requests, failing to match the high-throughput rhythm of arrayed computing units.

[0031] Therefore, there is an urgent need for a cache access mechanism adapted to this spatial parallel processor architecture to further reduce access latency and resource overhead on the basis of fully cross-connected hardware, so as to fully unleash the parallel access potential of the processor.

[0032] The following describes a cache access method provided by an embodiment of this application.

[0033] Please see Figure 3 , Figure 3 This is a flowchart illustrating a cache access method provided in an embodiment of this application. The method is applied to, for example... Figure 2 In the spatial parallel processor 20 shown, such as Figure 3 As shown, the method includes: S301, the target computing unit sends a temporary command to the target level 1 cache indicating a missing location index, and simultaneously sends the target address information to the cache controller.

[0034] The target computing unit refers to any computing unit with cache access requirements. In this example, a cache access request is initiated through a mechanism of command splitting and dual-path parallel transmission, which perfectly adapts to the special architecture of the spatial parallel processor and breaks through the timing limitations of traditional serial access.

[0035] Specifically, when a target computing unit initiates a cache access request, it does not need to wait for the complete command to be generated. Instead, it splits the cache access request into two independent parts and sends them out in parallel through different transmission paths: The first part is a temporary command, which carries the core operation information and corresponding data payload required to perform cache access. It covers fields necessary for the L1 cache to perform pre-startup operations, such as operation type, effective byte range, and inline access offset. This temporary command lacks a location index, so it does not yet have the complete conditions to perform the final storage access, but it is sufficient to drive the L1 cache storage array to complete the preliminary hardware preparation.

[0036] The second part is the target address information, which carries the complete address resolution field corresponding to the target data. It is sent to the cache controller for subsequent address decoding, storage resource matching, and location index generation.

[0037] The location index refers to the identifier of the storage resource corresponding to the target data indicated by the cache access request. It is a necessary field for performing cache access operations. This field cannot be directly generated by the computing unit and must be dynamically determined by the cache controller after completing address comparison, resource scheduling, and other control processes. This solution separates the location index from the full memory access command, allowing the first-level cache to receive temporary commands and start hardware preparation in advance without waiting for the cache controller's control process to complete. This fundamentally reduces the access latency caused by traditional serial access.

[0038] S302, in response to the temporary command, the target first-level cache performs a pre-startup operation. At the same time, the cache controller performs a parallel acquisition operation for the location index based on the target address information.

[0039] The pre-startup operation performed by the L1 cache is a preliminary hardware preparation for this cache access. This type of operation only relies on basic information such as the operation type and access granularity carried by the temporary command, and does not require locating the index as a prerequisite for execution. Specifically, after receiving the temporary command, the target L1 cache can complete one or more preparatory tasks in advance, such as memory pre-charging, access path activation, and data register pre-setting. In the traditional serial access process, this part of the work requires waiting for the complete command from the computing unit, resulting in a fixed startup delay. However, this solution reduces access latency by triggering the pre-startup operation in advance through the temporary command.

[0040] Meanwhile, the cache controller independently and in parallel executes the process of acquiring the location index. As a globally unified cache resource management node, the cache controller receives the target address information and independently completes control processes such as address decoding, storage resource matching, and scheduling allocation, ultimately generating a location index that can uniquely lock the target storage unit. This control process and the pre-startup process are completely overlapping in timing and independent of each other, realizing the parallel execution of the control process and the pre-startup process. At the same time, it can also adapt to the globally unified scheduling requirements of multiple computing units and multiple cache nodes under the spatial parallel processor architecture, avoiding hardware redundancy caused by repeated deployment of control logic on each cache node.

[0041] S303, the cache controller generates a narrow command carrying the positioning index and sends the narrow command to the target first-level cache.

[0042] The narrow command only carries a location index field and a matching command identifier, so its command width is much smaller than that of a temporary command that carries complete memory access information. Compared to the traditional architecture's mode of transmitting full memory access commands, this solution only needs to transmit narrow commands with extremely small bit widths on the control return path, eliminating the need to lay additional signal lines between the computing unit and the L1 cache to support the transmission of complete commands, thus reducing wiring costs.

[0043] The narrow command carries a command matching identifier that corresponds one-to-one with temporary commands stored in the target first-level cache, ensuring accurate matching between the location index and the corresponding access request during subsequent command completion and avoiding command confusion in scenarios with multiple concurrent requests. The timing of the cache controller generating the narrow command and sending it to the target first-level cache is aligned with the timing of the first-level cache's pre-startup operation. In some embodiments, when the narrow command arrives at the target first-level cache, the target first-level cache's pre-startup operation has already been completed or is nearly completed.

[0044] S304, the target first-level cache completes the temporary command according to the narrow command, generates a complete cache access command, and executes the corresponding cache access operation.

[0045] When the target first-level cache receives a narrow command from the cache controller, it uses the command matching identifier carried in the narrow command to locate the corresponding temporary command from multiple temporary commands temporarily stored in the local temporary queue, completes the command, and generates a complete cache access command. This process is completed locally in the target first-level cache, without the need to transmit additional information, resulting in extremely low hardware processing overhead and very short processing time.

[0046] After command completion, based on the complete cache access command, the storage array, which has already completed pre-booting, is driven to execute the read or write operation corresponding to this request. Since the pre-booting operation has been completed, when the complete command is generated, there is no need to repeat the pre-charge, path connection and other preparatory steps. It directly enters the core execution stage of data reading or writing, completely eliminating the serial waiting time in the traditional access process.

[0047] As can be seen, in this embodiment, by sending a temporary command lacking a location index to the target L1 cache and simultaneously sending the target address information to the cache controller in parallel, the target L1 cache can perform a pre-boot operation in advance, while the cache controller executes the location index acquisition process in parallel. This parallelizes the pre-boot of the storage array and the acquisition of the location index, reducing the inherent latency of the traditional serial access process. The cache controller generates a narrow command carrying the location index and sends it to the target L1 cache. The target L1 cache completes the temporary command based on the narrow command and executes the corresponding cache access operation. Since the bit width of the narrow command is much smaller than that of the temporary command, there is no need to lay out additional signal lines to support the transmission of the complete command, reducing wiring costs and transmission overhead. This is perfectly adapted to the spatial parallel processor architecture and improves the task processing efficiency of the spatial parallel processor.

[0048] In one possible example, the temporary command is a temporary read command, the target address information includes a target cache tag, and the cache controller performs a parallel acquisition operation for the location index based on the target address information, including: the cache controller determines a target cache path based on the target cache tag, the target cache path refers to the storage resource in the target first-level cache used to store the target cache line, the target cache line stores the target data; and determines the number of the target cache path as the location index.

[0049] In a caching system, a cache line is the smallest unit of data storage, consisting of a fixed-length, contiguous block of data. Data exchange between the Level 1 and Level 2 caches is performed on a per-cache-line basis, with each cache line storing multiple bytes of data at contiguous addresses. The target data for this read operation is contained within the corresponding target cache line.

[0050] In a set-associative cache architecture, a cache path is an independent branch of the storage array. Multiple independent cache paths can be mapped to the same address index. Each cache path can independently store one cache line and has its own independent read / write access path. Multiple cache paths together constitute a single level-one cache storage array, which is the core storage resource for read operations.

[0051] The cache tag is a high-order identifier field of the memory access address, used to uniquely distinguish cache lines from different addresses. Each cache path stores a corresponding cache tag, and it can be determined whether a cache path stores the target cache line by comparing the target cache tag with the cache tags stored in each cache path.

[0052] The temporary read command already contains all the basic information required for the L1 cache to complete the pre-startup of a read operation. This includes: a read operation type field to identify this access operation as a read operation, ensuring the L1 cache matches the pre-startup process under a read operation; an inline access offset and byte mask field to indicate the starting position of the target data within the cache line and the effective range of accessed bytes; a command matching identifier field to indicate the unique identifier of this access operation, which is bound one-to-one with the narrow commands subsequently issued by the cache controller to avoid command confusion; and a source computation unit identifier field to mark the initiator of this access request, i.e., the target computation unit. All of the above basic information are general attribute information for read operations, independent of specific cache path location results, and sufficient to support the target L1 cache in completing pre-startup work unrelated to storage array addressing.

[0053] The cache controller parses the target cache tag from the target address information, and then completes the matching and determination of the target cache path based on the target cache tag. The target cache path is the storage resource in the target first-level cache used to carry the target cache line. Its number is determined as the positioning index of this read operation, providing a precise positioning basis for the cache access operation executed after command completion.

[0054] The matching and determination process of the target cache path is executed independently and in parallel by the cache controller, which overlaps with the pre-boot operation of the target L1 cache in terms of timing, making full use of the parallel data path of the spatial parallel processor.

[0055] As can be seen, in this example, in the read operation scenario, the cache controller determines the target cache path based on the target cache tag and uses the number of the target cache path as the positioning index. This provides a precise positioning basis for the cache access operation executed after command completion. Moreover, the entire process is executed independently and in parallel by the cache controller, eliminating the serial latency of the traditional access process.

[0056] In one possible example, the cache controller determines the target cache path based on the target cache tag, including: the cache controller determining whether there is a cache path in the target first-level cache with the cache tag being the target cache tag; if it exists, then determining that cache path is the target cache path; if it does not exist, then determining whether there is an active entry for the target cache line in the state management entries, the active entry being used to indicate that the target cache line is being backfilled to the specified cache path of the target first-level cache; if it exists, then determining that specified cache path is the target cache path; if it does not exist, then determining the target cache path from the target first-level cache.

[0057] In this process, after parsing the target cache tag from the target address information, the cache controller traverses all cache tags of all cache paths under the corresponding cache group of the target first-level cache and matches them one by one to determine whether there is a cache path whose cache tag is the target cache tag.

[0058] If a matching cache path exists, a cache hit occurs, indicating that the target cache line has been completely stored in that cache path. You can then directly identify that cache path as the target cache path to trigger subsequent read operations.

[0059] If no matching cache path exists, a cache miss occurs. The cache controller then queries the status management entries to determine if there is an active entry for the target cache line. The status management entries are entries in the Miss Status Holding Register (MSHR), centrally managed by the cache controller, used to record all memory access requests that have triggered cache backfilling but are not yet completed. These entries include core fields such as the target cache line address, the allocated target cache path number, and the backfilling progress. An active entry is an entry in the MSHR that is in a valid registration state. It indicates that the target cache line is being backfilled from the second-level cache to the specified cache path in the first-level cache; that is, the corresponding cache path has completed resource allocation, the backfilling process has started, but the cache line data has not yet been fully written and is ready.

[0060] If an active entry for the target cache line exists, it means that a previous read request has triggered cache backfilling, the target cache line is currently in the backfilling process, and the corresponding cache path has been pre-allocated. In this case, the specified cache path marked by the active entry is designated as the target cache path. This eliminates the need for redundant cache resource allocation and repeated backfilling requests, effectively avoiding redundant resource calls and bandwidth waste caused by multiple requests to the same address in high-concurrency scenarios, thus improving the processing efficiency of concurrent access. Furthermore, after designating the specified cache path as the target cache path, the process further includes: adding the relevant information of the temporary read command to the active entry as another read request to access the target cache line. And after backfilling is completed and a cache access operation is triggered, the active entry is returned as a whole result to the temporary queue of the first-level cache, where the first-level cache unfolds the data reading of multiple read requests according to the request order.

[0061] If no active entry exists for the target cache line, it means the target cache line is not stored in the target first-level cache and is not in the refill process, which is a completely new cache missing scenario. The cache controller selects a cache path as the Victim Way from the corresponding cache group according to the preset replacement policy, determines the Victim Way as the target cache path, and synchronously allocates active entries, sending a data refill request to the target second-level cache.

[0062] As can be seen in this example, the cache controller dynamically determines the target cache path based on the cache tag comparison results and state management entries, and executes it in parallel with the pre-startup operation of the target first-level cache, with overlapping timing, thus eliminating the serial latency of the traditional access process.

[0063] In one possible example, determining the target cache path from the target first-level cache includes: determining whether there is an idle cache path in the target first-level cache with a read occupancy counter value of zero, wherein the read occupancy counter value is used to characterize the number of read requests occupying the target cache path; if it exists, then the idle cache path is determined as the target cache path; if it does not exist, then the read occupancy counter values ​​of each cache path in the target first-level cache are monitored until any cache path has a read occupancy counter value of zero, and the corresponding cache path is determined as the target cache path.

[0064] The read occupancy counter is an independent counter maintained by the cache controller for each cache path. Its value represents the number of read requests currently occupying that cache path. A counter value of 0 indicates that the cache path is not occupied by read requests, while a counter value of non-zero indicates that the cache path is occupied by read requests. In this case, it cannot be replaced as a sacrificial path to avoid data errors. Specifically, "occupied by read requests" means that the cache path is being backfilled or being read. That is, once a specific cache path is determined as a target cache path, its read occupancy counter value will not be zero until all data read operations corresponding to the target cache line it carries have been completed, and it cannot be replaced. Based on this, after the cache controller determines the target cache path according to the target cache tag, the method further includes: incrementing the read occupancy counter value of the target cache path by one to indicate that the target cache path is occupied by a new read request.

[0065] In this example, based on the scheduling logic of idle priority and waiting for release, for a new cache miss scenario, the system prioritizes matching idle cache paths with a read occupancy counter value of 0 in the target first-level cache, and directly starts the subsequent cache access and data backfilling process without waiting for the existing operations to finish, ensuring low access latency. If the read occupancy counter values ​​of all cache paths in the target first-level cache are greater than 0, it indicates that there are no idle resources available for immediate allocation. At this time, the system enters a polling monitoring state, continuously tracking the read occupancy counter values ​​of each cache path. When the read occupancy counter value of any cache path decreases to 0, that cache path is immediately identified as the target cache path, and its read occupancy counter value is updated from 0 to 1.

[0066] As can be seen in this example, in the scenario where the cache is missing and there are no active entries for the target cache line, the read occupancy counter value is used as the basis for cache path allocation. This achieves orderly scheduling of cache path resources in extremely high-concurrency scenarios, avoiding access conflicts and timing disorder caused by disordered contention among multiple requests.

[0067] In one possible example, after determining the target cache path from the target first-level cache, the method further includes: the cache controller sending a data backfill request to the target second-level cache; the target second-level cache responding to the data backfill request transmitting the target cache line to the target second-level cache interface; the target second-level cache interface backfilling the target cache line into the target cache path via a data transmission bypass, the data transmission bypass including a first data path and a second data path, the first data path being the data path between the target first-level cache and a specified computing unit, the second data path being the data path between the specified computing unit and the target second-level cache interface, and the computing unit array to which the specified computing unit belongs being the computing unit array corresponding to the target second-level cache interface.

[0068] Data backfilling is the process of transferring a target cache line from the target L2 cache to the target L1 cache. The cache controller can determine the target L2 cache based on the partition index segment in the target address information. In traditional processor architectures, the L1 cache and L2 cache are spatially close to each other and have a direct connection, so the data backfilling path is naturally from the L2 cache to the L1 cache. However, in the spatial parallel processor involved in this application embodiment, there is a computing unit array between the L1 cache and the L2 cache. Although a small number of wires are laid between them for transmitting some necessary commands, they are negligible compared to the normal number of wires and cannot support the data scale corresponding to data backfilling. If the traditional mode is to be reused in the spatial parallel processor, the number of wires between each L2 cache interface and each L1 cache needs to be increased to a scale that can support data backfilling, which will significantly increase the on-chip wiring complexity and hardware cost.

[0069] To address the aforementioned issues, this example constructs a data transfer bypass between the L1 and L2 cache interfaces by reusing the existing data paths of the computing unit. This enables data transfer between the L1 and L2 caches without additional wiring, making it suitable for spatially parallel processor distributed array architectures.

[0070] For example, such as Figure 4As shown, after allocating the target cache path in a cache miss scenario, the cache controller 21 synchronously sends a data backfill request to the target secondary cache 40, initiating the cache backfill process. Upon receiving the data backfill request, the target secondary cache 40 reads the target cache line at the corresponding address from its own storage array and transmits the entire line of data to the corresponding target secondary cache interface 241 for temporary storage. The target secondary cache interface 241 sends the temporarily stored target cache line to the designated computing unit 231 in the same array via the second data path 42. The designated computing unit 231 then routes and forwards the data. This path is a direct connection to the array, resulting in low transmission latency and stable bandwidth. Upon receiving the target cache line, the designated computing unit 231 transmits the entire line of data directly to the allocated target cache path via the first data path 41 between itself and the target primary cache 221, completing the data backfill operation. This path is an existing memory access data path, adapting to the global access characteristics of spatially parallel processors.

[0071] As can be seen, in this example, by splicing the existing data paths on both sides of the computing unit into a data transmission bypass, the cache line backfilling from any L2 cache interface to any L1 cache can be achieved without adding additional interconnect hardware, effectively controlling hardware costs. At the same time, the backfill data stream is transmitted separately from the regular computing access data stream, which will not occupy the access bandwidth of normal computing tasks, avoid the backfill operation from interfering with the parallel computing performance, and adapt to the high-concurrency computing characteristics of spatial parallel processors.

[0072] In one possible example, the temporary command is a temporary write-through command, the target address information includes a partition index segment, and the cache controller performs a parallel acquisition operation for the location index based on the target address information, including: the cache controller determines the target secondary cache interface based on the partition index segment; based on the target cache line corresponding to the target data, it determines a target aggregation buffer from multiple aggregation buffers of the target secondary cache interface, the target aggregation buffer being used to batch write all write data corresponding to the target cache line into the target secondary cache; and the number of the target aggregation buffer is determined as the location index.

[0073] A write-through operation refers to the operation of directly writing target data into the storage array of the next-level cache (such as the second-level cache). The temporary write-through command already contains all the basic information required for the first-level cache to complete the pre-startup of the write-through operation, such as: a write-through operation type field, which indicates that this access operation is a write-through operation, enabling the first-level cache to match the pre-startup process under the write-through operation, and to pre-open the write data path and configure the write driver timing; an inline access offset and byte mask field, which indicates the starting position of the data to be written in the target cache line and the effective range of bytes to be written, enabling the first-level cache to pre-complete the bit width alignment configuration and byte write path selection of the write data; a payload field, which carries the target data to be written in this write-through operation; a command matching identifier field, which indicates the unique identifier of this access operation and is bound one-to-one with the narrow commands issued by the cache controller to avoid command confusion in multi-concurrency scenarios; and a source computation unit identifier field, which marks the initiator of this access request, i.e., the target computation unit, so that the first-level cache can pre-reserve the status feedback path after the write-through operation is completed. The above basic information are all general attribute information for write-through operations, which do not depend on the specific aggregation buffer location results and are sufficient to support the target first-level cache to complete the preparatory work that is unrelated to the target buffer resource addressing.

[0074] The cache controller parses the partition index segment from the target address information, and then determines the target second-level cache and the target second-level cache interface corresponding to this write-through operation based on the partition index segment. After locking the target second-level cache interface, the cache controller determines the target aggregated buffer from multiple aggregated buffers within the interface according to the target cache line corresponding to the target data, and uses its number as the positioning index for this write-through operation. Then, it generates a narrow command and sends it to the target first-level cache to support command completion and the final execution of the write-through operation.

[0075] The aggregation buffer is used to write all the write data corresponding to the aggregation cache line into the second-level cache in batches, thereby adapting to the characteristics of multiple computing units in the spatial parallel processor architecture that can execute the writing of different data in the same cache line in parallel, and giving full play to the bandwidth optimization effect.

[0076] As can be seen, in this example, in the write-through operation scenario, the cache controller determines the target secondary cache interface based on the target address information and the target aggregation buffer based on the target cache line. The number of the target aggregation buffer is determined as the positioning index, providing a precise positioning basis for the cache access operation executed after command completion. Moreover, the entire process is executed independently and in parallel by the cache controller, eliminating the serial latency of the traditional access process.

[0077] In one possible example, determining the target cluster buffer from multiple cluster buffers of the target secondary cache interface based on the target cache line corresponding to the target data includes: determining whether there is an active buffer that matches the target cache line among the multiple cluster buffers of the target secondary cache interface; if there is, determining the active buffer as the target cluster buffer; if there is no active buffer, allocating an idle buffer for the target cache line and determining it as the target cluster buffer.

[0078] The active buffer refers to an aggregation buffer that is bound to a target cache line and is dedicated to the aggregation of data for that target cache line. It already contains some write data corresponding to the target cache line and can continuously receive new write data from the same cache line before the entire row of data aggregation is completed and batch writing is triggered. It is the core carrying state of the write aggregation mechanism. The idle buffer refers to an aggregation buffer that is not bound to any cache line and is in an idle and allocable state. It contains no valid write data and can be allocated to any new cache line write request as a new write aggregation carrying unit.

[0079] In this example, the cache controller queries the binding status of all aggregated buffers under the target second-level cache interface to check if there is an active buffer that matches the current target cache line. If it exists, it means that there are already write requests for the same cache line being aggregated, and the active buffer is directly identified as the target aggregated buffer, and the current write request and write data are aggregated together. If it does not exist, it means that this is the first direct write request for this cache line. At this time, the cache controller allocates an idle buffer from the idle buffer resource pool of the target second-level cache interface, binds it to the current target cache line, switches it to the new active buffer, and identifies it as the target aggregated buffer to carry all write data of the current and subsequent cache lines.

[0080] As can be seen, in this example, the cache controller determines the target aggregation buffer through status judgment and resource allocation, completes the location of storage resources, and obtains the location index. Moreover, the above status judgment and resource allocation processes are all completed independently by the cache controller based on the locally maintained buffer status information, and are executed in parallel with the direct write pre-start operation on the target first-level cache side, eliminating the serial latency of the traditional access process.

[0081] In one possible example, the target first-level cache completes the temporary command according to the narrow command to generate a complete cache access command and executes the corresponding cache access operation, including: the target first-level cache fills the target aggregation buffer number into the corresponding field of the temporary direct write command to generate a complete direct write command; the target first-level cache transmits the complete direct write command and the target data to the target aggregation buffer of the target second-level cache interface through data transmission bypass.

[0082] In this example, the data transmission bypass for write-through data is the same multiplexed path as the data transmission bypass in the aforementioned cache backfill scenario. This includes a first data path between the target L1 cache and the designated computing unit, and a second data path between the designated computing unit and the target L2 cache interface. This data transmission path fully reuses existing data paths, eliminating the need for additional transmission lines. While achieving cross-array write data scheduling, it effectively controls the hardware area and wiring complexity of on-chip interconnects.

[0083] For example, such as Figure 5 As shown, the target computing unit 232 sends a temporary direct write command to the target first-level cache 221, and simultaneously sends target address information to the cache controller 21 in parallel; the target first-level cache 221 performs a pre-boot operation, while the cache controller 21 performs a location index acquisition operation in parallel; the cache controller 21 generates a narrow command carrying the location index and sends the narrow command to the target first-level cache 221; the target first-level cache 221 completes the temporary direct write command according to the narrow command to generate a complete direct write command; the target first-level cache 221 transmits the direct write command and target data to the designated computing unit 231 through the first data path 41, and the designated computing unit 231 transmits the received direct write command and target data to the target aggregation buffer 2411 in the target second-level cache interface 241 through the second data path 42; after completing the aggregation of all data corresponding to the target cache line, the target aggregation buffer 2411 writes the data in batches to the target second-level cache 40. In this way, a complete transmission chain is constructed, from the target computing unit 232 to the target first-level cache 221, then to the designated computing unit 231, then to the target aggregation buffer 2411, and finally to the target second-level cache 40. The entire transmission chain does not require the deployment of additional data paths, effectively controlling hardware area and wiring complexity. It is understandable that although the transmission path of the target data passes through the target first-level cache, it is not actually written to the target first-level cache's storage array; it only occupies the target first-level cache's temporary queue and data path, without needing to activate the first-level cache's storage array.

[0084] As can be seen in this example, in the direct write scenario, the target first-level cache generates a complete direct write command by completing the temporary direct write command based on the narrow command, and reuses the data transmission bypass to complete the directional forwarding of write data to the target aggregation buffer. This retains the core optimization effects of command splitting and parallel execution, while controlling hardware costs through path reuse, and ensuring the efficient execution of direct write operations under the spatial parallel processor architecture.

[0085] In addition to the read and write scenarios described in the above embodiments, the cache access method described in this application is also suitable for ordinary write operation scenarios, namely, write operations that write target data to the target first-level cache's own storage array. The write data directly updates the target cache line in the first-level cache without needing to synchronize with the lower-level cache in real time. Since ordinary write operations are similar to read operations, their corresponding missing location index is the number of the target cache path. They also follow the core process of command splitting, parallel transmission, parallel execution of control flow and pre-startup process, command completion, and cache access, which will not be elaborated here.

[0086] This application also provides a computer storage medium storing a computer program or instructions thereon, which, when executed by a processor, implements some or all of the steps of any of the methods described in the above method embodiments.

[0087] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions within the scope of the technical solutions disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

[0088] While this application discloses the above information, it is not limited thereto. Any person skilled in the art can easily conceive of variations or substitutions without departing from the spirit and scope of this application, and can make various alterations and modifications, including combinations of the different functions and implementation steps described above, as well as software and hardware implementation methods, all of which are within the protection scope of this application.

Claims

1. A cache access method, characterized in that, The method is applied to a spatial parallel processor, which includes a cache controller, a level 1 cache array, multiple computing unit arrays, and multiple level 2 cache interfaces. Each computing unit array corresponds one-to-one with each level 2 cache interface. Each computing unit is connected to its corresponding level 2 cache through the level 2 cache interface of its respective computing unit array. Each computing unit is interconnected with each level 1 cache. The target computing unit sends a temporary command to the target first-level cache indicating a lack of a location index. At the same time, it sends the target address information to the cache controller in parallel. The location index refers to the identifier of the storage resource corresponding to the target data indicated by the cache access request. In response to the temporary command, the target first-level cache performs a pre-startup operation. At the same time, the cache controller performs a parallel acquisition operation for the location index based on the target address information. The cache controller generates a narrow command carrying the location index and sends the narrow command to the target first-level cache. The bit width of the narrow command is much smaller than the bit width of the temporary command. The target first-level cache completes the temporary command based on the narrow command, generates a complete cache access command, and executes the corresponding cache access operation.

2. The method according to claim 1, characterized in that, The temporary command is a temporary read command, the target address information includes a target cache tag, and the cache controller performs a parallel acquisition operation for the location index based on the target address information, including: The cache controller determines the target cache path based on the target cache tag. The target cache path refers to the storage resource in the target first-level cache used to store the target cache line, and the target cache line stores the target data. The number of the target cache path is determined as the location index.

3. The method according to claim 2, characterized in that, The cache controller determines the target cache path based on the target cache tag, including: The cache controller determines whether there is a cache path in the target first-level cache with the cache tag of the target cache tag; If it exists, then the cache path is determined to be the target cache path; If it does not exist, then determine whether there is an active entry for the target cache line in the state management entries. The active entry is used to indicate that the target cache line is being backfilled into the specified cache path of the target first-level cache. If it exists, then the specified cache path is determined to be the target cache path; If it does not exist, then the target cache path is determined from the target first-level cache.

4. The method according to claim 3, characterized in that, Determining the target cache path from the target first-level cache includes: Determine whether there is an idle cache path in the target first-level cache with a read occupancy counter value of zero. The read occupancy counter value is used to represent the number of read requests occupying the target cache path. If it exists, then the free cache path is determined as the target cache path; If it does not exist, monitor the read occupancy counter value of each cache path in the target first-level cache until the read occupancy counter value of any cache path is zero, and determine the corresponding cache path as the target cache path.

5. The method according to claim 4, characterized in that, After determining the target cache path from the target first-level cache, the method further includes: The cache controller sends a data refill request to the target secondary cache; In response to the data backfill request, the target second-level cache transmits the target cache line to the target second-level cache interface; The target secondary cache interface backfills the target cache line into the target cache path through a data transmission bypass. The data transmission bypass includes a first data path and a second data path. The first data path refers to the data path between the target primary cache and the designated computing unit. The second data path refers to the data path between the designated computing unit and the target secondary cache interface. The computing unit array to which the designated computing unit belongs refers to the computing unit array corresponding to the target secondary cache interface.

6. The method according to claim 1, characterized in that, The temporary command is a temporary write-through command, the target address information includes a partition index segment, and the cache controller performs a parallel acquisition operation for the location index based on the target address information, including: The cache controller determines the target second-level cache interface based on the partition index segment; Based on the target cache line corresponding to the target data, a target aggregation buffer is determined from multiple aggregation buffers of the target second-level cache interface. The target aggregation buffer is used to batch write all write data corresponding to the target cache line into the target second-level cache. The number of the target aggregation buffer is determined as the positioning index.

7. The method according to claim 6, characterized in that, The step of determining the target cluster buffer from multiple cluster buffers of the target second-level cache interface based on the target cache line corresponding to the target data includes: Determine whether there is an active buffer that matches the target cache line among the multiple aggregated buffers of the target second-level cache interface; If it exists, then the active buffer is determined to be the target aggregation buffer; If it does not exist, then allocate a free buffer for the target cache line and determine it as the target aggregation buffer.

8. The method according to claim 6 or 7, characterized in that, The target first-level cache completes the temporary command based on the narrow command to generate a complete cache access command, and executes the corresponding cache access operation, including: The target first-level cache fills the target aggregation buffer number into the corresponding field of the temporary direct write command to generate a complete direct write command; The target first-level cache transmits the complete write command and the target data to the target aggregation buffer of the target second-level cache interface through a data transmission bypass.

9. A spatial parallel processor, characterized in that, The spatial parallel processor is used to execute the method as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1-8.

Images