Encapsulation structure and data access methods

Abstract

This application provides a packaging structure and data access method applicable to the field of hardware technology. The packaging structure includes a substrate, a processing layer, a cache layer, an interconnect layer, and a global cache controller, wherein: the interconnect layer and the global cache controller are located on the substrate and are interconnected through physical channels on the substrate; the processing layer is located on the side of the interconnect layer away from the substrate and is connected to the interconnect layer; the cache layer is located on the side of the global cache controller away from the substrate and is connected to the global cache controller; the global cache controller is used to perform access operations on the cache layer according to access requests provided by the processing layer, and returns response data generated by the cache layer based on the access operations to the processing layer; the interconnect layer is used to route data transmission between the processing layer and the global cache controller.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hardware, and more particularly to a packaging structure and a data access method. BACKGROUND

[0002] Chiplet technology, as a key path to continue Moore's law, splits complex processor systems into multiple functionally independent and individually manufactured and tested small chips, such as computing chiplets, cache chiplets, and interconnection chiplets, through modularization and advanced packaging integration, and then uses 2.5D or 3D packaging technology, such as an interposer or a silicon bridge, to achieve high-density integration. Chiplet technology makes it possible to build a multi-processor chiplet cluster due to its high-density integration characteristics, but the scalability of the cross-chiplet shared cache scheme has become a key bottleneck restricting further performance improvement. SUMMARY

[0003] According to a first aspect of the present application, a packaging structure is provided, which includes a substrate, a processing layer, a cache layer, an interconnection layer, and a global cache controller. The interconnection layer and the global cache controller are located on the substrate and connected to each other through physical channels on the substrate. The processing layer is located on the side of the interconnection layer away from the substrate and connected to the interconnection layer. The cache layer is located on the side of the global cache controller away from the substrate and connected to the global cache controller. The global cache controller is configured to perform access operations on the cache layer according to access requests provided by the processing layer and return response data generated by the cache layer based on the access operations to the processing layer. The interconnection layer is configured to route data transmission between the processing layer and the global cache controller.

[0004] A second aspect of the present application provides a data access method, which is executed by the packaging structure described above. The data access method includes: the processing layer generates an access request; the interconnection layer selects a target transmission path from multiple transmission paths between the processing layer and the global cache layer according to the access request from the processing layer, and provides the access request to the global cache controller through the target transmission path; the global cache controller performs access operations on the cache layer according to the access request provided by the interconnection layer, obtains response data generated based on the access operations from the cache layer, and provides the response data to the interconnection layer; and the interconnection layer returns the response data provided by the global cache controller to the processing layer through the target transmission path.

[0005] According to embodiments of this application, the processing layer, cache layer, interconnect layer, and global cache controller can be physically integrated using 3D integrated packaging technology, integrating them onto the same substrate. The interconnect layer and global cache controller enable data transmission between the processing layer and the cache layer. The global cache controller allows the processing layer to access the cache layer and also centrally manages the cache resources of the cache layer, enabling cross-chip shared cache. Based on the scalable design of the processing layer, cache layer, and interconnect layer, the scalability of cross-chip shared cache in multi-processor chip clusters can be improved. Attached Figure Description

[0006] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments of this application with reference to the accompanying drawings.

[0007] Figure 1 A schematic diagram of the packaging structure according to an embodiment of this application is shown.

[0008] Figure 2 A schematic diagram of a 3D integrated packaging substrate according to an embodiment of this application is shown.

[0009] Figure 3 A schematic diagram of an interconnect layer according to an embodiment of this application is shown.

[0010] Figure 4 A schematic diagram of the processing layer according to an embodiment of this application is shown.

[0011] Figure 5 A schematic diagram of a global cache controller according to an embodiment of this application is shown.

[0012] Figure 6 A schematic diagram of a cache layer according to an embodiment of this application is shown.

[0013] Figure 7 A flowchart illustrating the operation of a data access method according to an embodiment of this application is shown.

[0014] Figure 8 A schematic diagram of a data access method according to an embodiment of this application is shown. Detailed Implementation

[0015] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0016] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0017] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0018] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0019] Core technology can not only reduce the manufacturing difficulty of individual cores, but also improve the yield rate and reduce costs through independent testing and screening. It can also flexibly combine cores with different processes and functions to meet diverse needs.

[0020] In a multi-processor chip cluster, the performance of the caching system directly determines the overall computing power. Frequent data interactions between processors rely on external memory, which can significantly reduce computational efficiency due to access latency ranging from tens to hundreds of nanoseconds. Therefore, connecting multiple processor chips via high-speed interconnect technology and sharing on-chip caches to form a cross-chip shared cache can reduce data interaction latency and improve cache resource utilization. By enabling data sharing based on proximity through a shared cache, the data access latency between processor chips can be reduced to the nanosecond level. Simultaneously, it can reduce resource waste caused by multiple processor chips repeatedly caching the same data, enhancing the parallel processing capability and energy efficiency of the multi-processor chip cluster's caching system.

[0021] Cross-chip shared cache solutions for multi-processor chip clusters in related technologies include shared cache designs based on multi-chip modules, cache coherence architectures based on non-standard interconnect protocols, and chip integration solutions based on 2.5D or 3D packaging technologies.

[0022] Shared cache designs based on multi-chip modules integrate multiple processor chips with independent shared cache chips via a packaging substrate. They utilize traditional buses such as PCIe (Peripheral Component Interconnect Express) and AXI (Advanced Xtensible Interface), or crossbar switches, to interconnect the multiple processor chips. Cache coherency is maintained through directory protocols or snooping protocols. For example, some server processors integrate 2 to 4 processor core chips with 1 shared L3 or L4 cache chip using MCM (Multi-Chip Module) technology. Cache coherency relies on the snooping mechanism of the processor's internal bus and only supports small-scale chip clusters typically not exceeding 8 core chips.

[0023] The cache coherence architecture based on non-standard interconnect protocols achieves cache coherence access between the processor and the 2D coplanar through protocol adaptation. Due to limitations of resources such as directories, it only supports small-scale expansion.

[0024] Chip integration solutions based on 2.5D or 3D packaging technology utilize high-density interconnects in silicon interposers to achieve low-latency connections between chips, integrating processor chips, cache chips, and interconnect chips within the same package. For example, some high-end chips employ 3D packaging technology, stacking multiple layers of on-chip cache chips on top of the processor chip, achieving nanosecond-level data interaction through through-silicon vias (TSVs). Cache consistency is maintained by an integrated directory controller within the chip. However, limitations in the number of stacked layers and interconnect density restrict the scale of cache resource pooling and chip expansion capabilities.

[0025] Cross-chip shared cache schemes for multi-processor chip clusters in related technologies have several technical drawbacks. Traditional bus or cross-switch interconnects experience a significant increase in latency as the number of chips increases, making bandwidth a bottleneck and limiting scalability, thus hindering support for large-scale processor arrays and cache resource pooling. Heterogeneous protocols involve numerous redundant operations and high consistency message overhead during cross-chip cache access, resulting in low protocol efficiency and difficulty in meeting the demands for low-latency, high-throughput data access. Resource management and fault recovery mechanisms are insufficient; the lack of a globally unified cache management strategy prevents dynamic address mapping, load balancing, and failover, leading to poor fault tolerance. Cache access paths are not adequately optimized; protocol conversion and data encapsulation introduce additional latency, and the lack of fine-grained energy efficiency control mechanisms results in a performance-energy efficiency imbalance, making it difficult to achieve both high performance and low power consumption.

[0026] Therefore, there is an urgent need for a cross-chip cache architecture with efficient interconnection, scalable resource sharing, intelligent management, and strong fault tolerance to support the needs of next-generation high-performance computing and artificial intelligence workloads.

[0027] This application proposes a packaging structure and data access method that enables efficient interconnection of multiple processor chips, establishes a unified cross-chip shared on-chip cache, and realizes global sharing and management of cache resources.

[0028] Figure 1 A schematic diagram of the packaging structure according to an embodiment of this application is shown.

[0029] like Figure 1 As shown, the package structure 100 includes a substrate 110, a processing layer 120, a cache layer 130, an interconnect layer 140, and a global cache controller 150. The processing layer 120, cache layer 130, and interconnect layer 140 are all individual chiplets. The processing layer 120 can be a processor chip capable of implementing computing functions. The cache layer 130 can be a cache chip for implementing cache functions. The interconnect layer 140 can be an interconnect chip capable of implementing interconnect functions.

[0030] Interconnect layer 140 and global cache controller 150 are located on substrate 110 and are interconnected through physical channels on substrate 110. Processing layer 120 is located on the side of interconnect layer 140 away from substrate 110 and is connected to interconnect layer 140. Cache layer 130 is located on the side of global cache controller 150 away from substrate 110 and is connected to global cache controller 150. Global cache controller 150 is used to perform access operations on cache layer 130 according to access requests provided by processing layer 120, and returns response data generated by cache layer 130 based on the access operations to processing layer 120. Interconnect layer 140 is used to route data transmission between processing layer 120 and global cache controller 150.

[0031] According to embodiments of this application, the processing layer 120, cache layer 130, interconnect layer 140, and global cache controller 150 can be physically integrated using 3D integrated packaging technology, integrating them onto the same substrate 110. The interconnect layer 140 and global cache controller 150 enable data transmission between the processing layer 120 and cache layer 130. The global cache controller 150 enables the processing layer 120 to access the cache layer 130 and can also uniformly manage the cache resources of the cache layer 130, achieving cross-chip shared cache. Based on the scalable design of the processing layer 120, cache layer 130, and interconnect layer 140, the scalability of cross-chip shared cache in multi-processor chip clusters can be improved.

[0032] The hierarchical scalable architecture, which adopts three types of chips—processing layer 120, cache layer 130, and interconnect layer 140—can break through the scalability bottleneck of traditional multi-chip modules (MCMs). The number of chips of each type can be flexibly configured as needed. At the same time, the global cache controller 150 is deployed at the interaction hub between interconnect layer 140 and cache layer 130 to manage cache resources in a unified manner, which can provide support for large-scale processor arrays and cache resource pooling.

[0033] Figure 2 A schematic diagram of a 3D integrated packaging substrate according to an embodiment of this application is shown.

[0034] like Figure 2 As shown, interconnect layer 140 and global cache controller 150 are located on the substrate and are connected to each other through physical channels on the substrate. Processing layer 120 is located on the side of interconnect layer 140 away from the substrate and is connected to interconnect layer 140. Cache layer 130 is located on the side of global cache controller 150 away from the substrate and is connected to global cache controller 150. The number of processing layer 120, cache layer 130, and interconnect layer 140 can be n, where n is a positive integer. The number of processing layer 120, cache layer 130, and interconnect layer 140 can also be different. Processing layer 120 can include multiple computing cores C. Processing layer 120 and interconnect layer 140 can be connected through vertical interconnect channel (TSV). Cache layer 130 and global cache controller 150 can be connected through vertical interconnect channel (TSV). Interconnect layer 140 and global cache controller 150 can be connected correspondingly through physical channels.

[0035] Figure 3 A schematic diagram of an interconnect layer according to an embodiment of this application is shown.

[0036] like Figure 3 As shown, the interconnection layer 140 includes multiple first ports 141, multiple second ports 142, and a routing controller 143, with multiple transmission paths existing between the multiple first ports 141 and the multiple second ports 142.

[0037] like Figure 1 and Figure 3 As shown, multiple first ports 141 are connected to the processing layer 120. Multiple second ports 142 are connected to the global cache controller 150. The routing controller 143 is configured to, in response to a target first port among the multiple first ports 141 receiving an access request from the processing layer 120, select a target second port from the multiple second ports 142 according to the access request, determine a target transmission path between the target first port and the target second port among multiple transmission paths, provide the access request to the global cache controller 150 through the target transmission path, and return the response data provided by the global cache controller 150 for the access request to the processing layer 120 through the target transmission path.

[0038] like Figure 1 and Figure 3 As shown, the first port 141 may include a local interface A, a vertical connection channel (TSV), and a traffic scheduler output port DC. The interconnect layer 140 receives access requests from the processing layer 120 through the local interface A and the vertical connection channel (TSV) of the first port 141, and pushes valid data frames from the access requests from the processing layer 120 to the traffic scheduler output port DC. The routing controller 143 can parse the valid data frames from the access requests from the processing layer 120, select the target second port, and transmit the valid data frames from the access requests to the target second port, providing them to the global cache controller 150 through the target transmission path. In one example, the routing controller 143 can parse the 16-bit identifier of the target second port from the access requests from the processing layer 120 within one clock cycle, and query the 256-node routing table based on the identifier of the target second port to determine the target second port. The routing table of the routing controller 143 can support 256 target nodes and can also effectively avoid network deadlock by using a virtual channel (VC) mechanism, for example, 8 virtual channels.

[0039] According to an embodiment of this application, the interconnection layer 140 can realize routing selection between the processing layer 120 and the global cache controller 150 through a plurality of first ports 141, a plurality of second ports 142 and a routing controller 143.

[0040] According to embodiments of this application, access requests are categorized into multiple types, each with corresponding priorities. The routing controller 143 is configured to determine the target transmission path between a target first port and a target second port among multiple transmission paths by allocating crossbar switch resources according to the priority of the access requests, based on a Weighted Fair Queuing (WFQ) scheduling algorithm.

[0041] The routing controller 143 can implement traffic scheduling and routing functions. The routing controller 143 can employ a weighted fair queue scheduling algorithm to allocate bandwidth based on priority and Quality of Service (QoS). The routing controller 143 may also include a 16×16 crossbar switch matrix, with each port of the 16×16 crossbar switch matrix equipped with a 128KB input / input buffer, and a credit-based flow control mechanism to prevent buffer overflow.

[0042] In one example, 16×16 crossbar switch resources can be allocated according to 8 QoS priority levels. A target transmission path is established within one clock cycle, forwarding access requests from processing layer 120 to the traffic scheduler input port DR of the second port 142. For example, QoS priority mapping could divide access requests into 3 priority categories, mapping these 3 categories to 8 QoS priority levels. For instance, cache consistency messages in access requests are classified as the highest priority, mapped to QoS priority levels 7-6; cache read / write requests are classified as medium priority, mapped to QoS priority levels 5-2; and control commands in access requests are classified as low priority, mapped to QoS priority levels 1-0, ensuring that critical cache consistency messages are transmitted first.

[0043] The route controller 143 can employ a distributed routing algorithm based on a fat tree topology, compatible with both source routing mode and adaptive routing mode. In source routing mode, the route controller 143 selects the destination second port based on the access request from the processing layer 120.

[0044] The routing controller 143 may also include a congestion control unit. Upon detecting congestion on the target second port, such as a occupancy rate greater than 70%, the congestion control unit can send a congestion notification to the sender. In response, the sender can reduce the data transmission rate, for example, by a step size of 10%, and simultaneously activate adaptive routing mode, selecting a backup second port 142 to offload some traffic to the second port 142 with a load rate less than 50%, thus dynamically adjusting the selection of the target second port. The congestion control unit can dynamically monitor the buffer occupancy rate to adjust the data rate, thereby effectively preventing network congestion.

[0045] The routing controller 143 may also include a fault detection unit that can monitor the link status of the target transmission path in real time. In the event of a failure in the target transmission path, it can automatically switch to the backup second port 142 to improve the reliability of data transmission.

[0046] like Figure 3 As shown, the interconnect layer 140 also includes a protocol adaptation unit 144. The second port 142 may include a flow scheduler input port DR, a physical layer interface (PHY), and a serial interface SD. The serial interface SD may be a serializer / deserializer (SerDes).

[0047] Protocol adaptation unit 144 is configured to convert the access request into a data format conforming to the target cache coherency protocol before providing the access request to the global cache controller 150 via the target transmission path. The target cache coherency protocol can be the CXL (Compute Express Link) protocol, which is a heterogeneous cache coherency protocol standard.

[0048] The data format of access requests conforming to the target cache consistency protocol can be divided into three transmission modes: cache read / write mode, cache consistency message mode, and control command mode. Cache read / write mode can be used for data read and write access from processing layer 120 to cache layer 130. Cache consistency message mode can be used to transmit consistency messages between the private cache of processing layer 120 and cache layer 130 to maintain data consistency. Control command mode can be used by the global cache controller 150 to configure, manage, and monitor the status of cache layer 130.

[0049] Protocol adaptation unit 144 can use a 32-bit CRC (Cyclic Redundancy Check) checksum algorithm to verify access requests, ensuring data accuracy. Protocol adaptation unit 144 can also employ Error Detection and Correction (EDC) methods for error detection and correction. Upon detecting a transmission error, protocol adaptation unit 144 can automatically trigger a retransmission mechanism based on the target cache consistency protocol to restore communication, further enhancing the reliability and stability of data communication. For example, protocol adaptation unit 144 can add a link layer header and a new CRC checksum to the transmitted data.

[0050] Protocol adaptation unit 144 can perform heterogeneous protocol conversion. When an access request from processing layer 120 does not support the target cache consistency protocol, protocol adaptation unit 144 can convert the access request into a data format that conforms to the target cache consistency protocol, which can improve the compatibility and correctness of access to shared caches across kernels.

[0051] like Figure 2 and Figure 3As shown, the serial interface SD includes 16 transceivers, each employing 4-channel PAM4 (Pulse Amplitude Modulation 4) technology. Each channel can achieve a data rate of 112Gbps, with an aggregate bandwidth of 448Gbps. The transceiver includes a transmitter and a receiver. The transmitter includes an 8:1 parallel-to-serial converter, PAM4 encoding, and pre-emphasis circuitry. The receiver includes a CTLE (Continuous-Time Linear Equalizer), a DFE (Decision Feedback Equalizer), and a CDR (Clock and Data Recovery Circuit). Combined with a link training module, automatic gain and equalization adjustment are implemented to ensure link robustness. Data transmission is ultimately achieved through 4 independent physical layer interfaces (PHY).

[0052] like Figure 2 and Figure 3 As shown, a process of 7nm or higher can be used to integrate a 16-channel high-speed serial interface SD at the edge of each interconnect layer 140. The interconnect layer 140 can integrate a routing controller 143 and a protocol adapter unit 144, which can realize routing, traffic scheduling and error correction functions, and can ensure efficient and reliable data transmission.

[0053] like Figure 1 and Figure 3 As shown, the internal data transmission path of the interconnect layer 140 is a bidirectional closed loop. The interconnect layer 140 receives response data from the cache layer 130 through the global cache controller 150, and transmits the response data from the cache layer 130 to the processing layer 120 through the reverse target transmission path. The routing controller 143 can receive access requests from the processing layer 120 through the local interface of the first port 141, the vertical connection channel TSV, and the traffic scheduler output port DC. Based on the access request, it determines the target second channel and establishes the target transmission path. The protocol adaptation unit 144 processes the access request and transmits it to the global cache controller 150 through the traffic scheduler input port DR, the physical layer interface PHY, and the serial interface SD of the second port 142, thereby transmitting it to the cache layer 130. The target transmission path sequentially includes the local interface, the vertical connection channel TSV, the traffic scheduler output port DC, the routing controller 143, the protocol adaptation unit 144, the traffic scheduler input port DR, the physical layer interface PHY, and the serial interface SD.

[0054] In one example, interconnect layer 140 is used to implement data transmission between all processing layers 120 and between processing layer 120 and cache layer 130. The interconnect network for data transmission can adopt a fat-tree topology, with interconnect layer 140 positioned in the middle, and processing layers 120 and cache layer 130 connected as leaf nodes. This interconnect network provides abundant redundant paths, improving network bandwidth and system reliability, and simplifying routing algorithm design. Interconnect layer 140 can connect to processing layer 120 via four first ports 141 and to cache layer 130 via a global cache controller 150 via eight second ports 142. The number of second ports 142 is greater than the number of first ports 141; for example, the number of second ports 142 is twice the number of first ports 141, which can meet the requirements of high-bandwidth data output.

[0055] Figure 4 A schematic diagram of the processing layer according to an embodiment of this application is shown.

[0056] like Figure 2 , Figure 3 and Figure 4 As shown, the processing layer 120 includes multiple local interconnect interfaces 121, which are connected to multiple first ports 141 of the interconnect layer 140 via multiple vertical interconnect channels (TSVs). The vertical interconnect channels (TSVs) can be vertical electrical links constructed using through-silicon via (TSV) technology. Figure 1 As shown, the processing layer 120 and the interconnect layer 140 can establish a connection via a TSV link, enabling direct interconnection between the layers and achieving low-latency connection between the processing layer 120 and the interconnect layer 140. The processing layer 120 may include a vertical path control unit, which controls the electrical connection of the vertical connection channel TSV.

[0057] The local interconnect interface 121 can send data generated within the processing layer 120 to the interconnect layer 140 via the first port 141, and can also receive data from other interconnect layers via the first ports 141 of other interconnect layers. For example, the first processing layer can transmit data to the first interconnect layer and receive data from the first interconnect layer via the connection between its local interconnect interface 121 and the first port 141 of the first interconnect layer. After completing a portion of the computation task, the first processing layer can also transmit the computation result to the second interconnect layer and receive data from the second interconnect layer via the connection between its local interconnect interface 121 and the first port 141 of the second interconnect layer, and continue computation on the data from the second interconnect layer.

[0058] like Figure 2 and Figure 4As shown, the processing layer 120 also includes multiple computing cores C, a first-level cache L1, a second-level cache L2, and a third-level cache L3. The cache layer 130 is a globally shared cache. The first-level cache L1, the second-level cache L2, and the third-level cache L3 can be private caches of the processing layer 120. The computing cores C are responsible for core computational tasks and reduce access latency through local third-level cache L3 in coordination with the cross-core globally shared cache.

[0059] The computing core C can be a high-performance core based on either ARM (Advanced RISC Machine) or RISC-V (Reduced Instruction Set Computing-V) architecture. Each computing core C can integrate 8-16 cores, with each core containing an integer / floating-point arithmetic unit, a branch predictor, an instruction decoder, etc.

[0060] Each compute core C is equipped with an independent Level 1 cache (L1). Multiple compute cores C, for example, four, can share a Level 2 cache (L2). The Level 1 cache (L1) can be an L1 cache (Level 1 Cache), which includes an L1 I-Cache (Level 1 Instruction Cache) and an L1 D-Cache (Level 1 Data Cache). The Level 2 cache (L2) can be an L2 cache.

[0061] Multiple compute cores C can share the third-level cache (L3). The third-level cache L3 can be an L3 cache (Level 3 Cache). The L3 cache can be a core-level L3 cache of 32MB or more, a multi-way set-associative shared cache, including a cache array, tag storage and replacement policy unit.

[0062] Cache layer 130 is a globally shared cache. The globally shared cache can be an L4 cache (Level 4 Cache).

[0063] like Figure 1 and Figure 4 As shown, the processing layer 120 also includes a first interface controller 122. The first interface controller 122 is connected between the third-level cache L3 and the local interconnect interface 121, and can be referred to as the L3-L4 interface controller. The first interface controller 122 is configured to send an access request for the globally shared cache to the interconnect layer 140 through at least one of the multiple local interconnect interfaces 121 in response to a query that fails to find a cache in the first-level cache L1, the second-level cache L2, and the third-level cache L3.

[0064] The first interface controller 122 can adopt a dual-port parallel architecture to separate read and write operations, integrate asynchronous processing of access request or response data with a depth of 16, and adapt and convert it to a local request format through protocols such as heterogeneous cache consistency protocols. By optimizing the interface logic of the third-level caches L3 and L4, efficient interaction of data between the local cache and the global cache is ensured.

[0065] The processing layer 120 sequentially sends access requests for the global shared cache to the interconnection layer 140 via the computing core C, the first-level cache L1, the second-level cache L2, the third-level cache L3, the first interface controller 122, and the local interconnection interface 121. Simultaneously, the local interconnection interface 121 can also receive response data from the global shared cache through the interconnection layer 140 and the global cache controller 150 to update the data in the third-level cache L3.

[0066] Figure 5 A schematic diagram of a global cache controller according to an embodiment of this application is shown.

[0067] like Figure 1 , Figure 3 and Figure 5 As shown, the global cache controller 150 includes multiple third ports 151, which are connected to multiple second ports 142 of the interconnect layer 140 via multiple physical channels on the substrate 110. The third ports 151 and 142 are serial interfaces supporting the target cache consistency protocol. The third ports 151 integrate 16 SerDes channels, which can support cache read / write request forwarding, consistency message transmission, and control command interaction. The third ports 151 can transmit consistency messages between the private cache of the processing layer 120 and the global cache of the cache layer 130, maintaining data consistency.

[0068] The cache layer 130 may include multiple cache units, which store data. The global cache controller 150 also includes a multi-core control cluster 152, a hardware acceleration module 153, and a storage unit.

[0069] The multi-core control cluster 152 is connected to multiple third ports 151 to receive access requests from the interconnect layer 140 through the multiple third ports 151, determine the cache unit to be accessed by querying the address mapping table according to the access request, perform access operation on the cache unit to be accessed, obtain response data generated based on the access operation from the cache unit to be accessed, and provide the response data to the interconnect layer 140.

[0070] The hardware acceleration module 153 is connected to the multi-core control cluster 152 and is used to provide hardware acceleration for address mapping table lookups of the multi-core control cluster 152.

[0071] Storage unit 154 is connected to multi-core control cluster 152 and is used to store address mapping tables. Storage unit 154 can also store cache directory tables, priority score caches, and non-volatile storage.

[0072] The multi-core control cluster 152 is also used to parse the access request based on the target cache consistency protocol after receiving the access request, and to package the response data based on the target cache consistency protocol before providing the response data to the interconnect layer 140, so as to achieve target cache consistency.

[0073] There are multiple cache layers 130 and multiple processing layers 120. For example... Figure 5 As shown, the multi-core control cluster 152 includes a first processor microcore M0, a second processor microcore M1, a third processor microcore M2, and a fourth processor microcore M3. Figure 5 As shown, the hardware acceleration module 153 may include at least one of the following: a computation accelerator 1531, a priority calculation accelerator 1532, and a directory synchronization accelerator 1533.

[0074] The first processor microcore M0 is used to divide the global address space for multiple cache layers 130 into multiple address blocks, each address block corresponding to a cache unit; in response to receiving an access request through the interconnect layer, it determines the target address block corresponding to the access request from the multiple address blocks, and determines the target cache layer where the target address block is located from the multiple cache layers 130 by querying the address mapping table; if the target cache layer is available, it forwards the access request to the interconnect layer 140; if the target cache layer is unavailable, it migrates the data in the target cache layer to the available cache layer 130 among the multiple cache layers 130, and sends a notification for updating the address mapping table to the multiple processing layers 120 through the interconnect layer 140.

[0075] The computation accelerator 1531 is used to generate an index of an address block by performing a hash calculation on the block's identification information. This index is then used for lookups in the address mapping table. The computation accelerator can be a hash accelerator, employing a parallel 32-bit CRC hash circuit, supporting 128-bit address input, and completing the hash value calculation within one clock cycle, thus providing hardware acceleration for address mapping table lookups.

[0076] The first processor microcore M0 may include CX protocol message parsing 1521.

[0077] The global cache controller 150 can implement address mapping management. It can divide the global address space for multiple cache layers 130 into multiple address blocks with a granularity of 64KB. Each address block is assigned a unique identifier, which can be a 16-bit binding identifier. The address mapping table can be 512MB in size, using a 2-way set-associative structure, storing the mapping relationship between "global address block - target cache layer - link status". The link status can be available, congested, or faulty, and it supports simultaneous read and write operations. When the processing layer 120 initiates an access request, the global cache controller 150 receives the target address block and calculates the 8-bit hash value of ABID (Address Binding Identifier) ​​using a hash accelerator. The hash function Hash(ABID) can be expressed as:

[0078] Hash(ABID)=(ABID × 0x5F3A + 0x12C7) mod 256 (1);

[0079] Here, ×0x5F3A represents multiplying by the hexadecimal constant 0x5F3A, and +0x12C7 represents adding the hexadecimal constant 0x12C7. mod 256 represents taking the modulo of 256, retaining the lower 8 bits of the result, and obtaining an 8-bit hash value. The hash function Hash(ABID) can be a hardware-optimized low-collision function.

[0080] Based on the hash value index address mapping table, obtain the address and link status of the target cache layer. If the link status is available, forward the request through the interconnection layer 140. If the link status is congested, trigger dynamic route adjustment.

[0081] The global cache controller 150 can poll the status signals of multiple cache layers 130 at 100ns intervals via a fault monitoring unit. The status signals of cache layers 130 can be a power good signal or an ECC (Error Correction Code) error count. If a cache layer 130 reports a fault three times consecutively, it is marked as "unavailable" and designated as a faulty cache layer. All ABID target address blocks managed by the faulty cache layer are extracted from the address mapping table and redistributed to other normal cache layers 130 according to their hash values. A "block migration instruction" can be issued to the cache layer 130 to copy valid data from the faulty cache layer to the target cache layer. For example, non-dirty blocks are discarded directly, while dirty blocks are written back to memory and then migrated. After the block migration is complete, the address block of the target cache layer corresponding to the ABID in the address mapping table can be atomically updated, and a "mapping table synchronization notification" is sent to all processing layers 120 to ensure that subsequent accesses point to the new target.

[0082] The second processor microcore M1 is used to maintain a cache directory for each of the multiple cache layers 130. The cache directory records the holders of each cache unit in the cache layer 130, including at least one of the multiple processing layers 120. Based on the cache directory of the target cache layer, the holder of the cache unit corresponding to the target address block is determined, and an invalidation command is sent to the holder through the interconnect layer 140. The invalidation command is used to invalidate the access rights of the holder's processing layer to the cache unit corresponding to the target address block.

[0083] The cache directory table includes cache directories for multiple cache layers 130. The cache directory table has a capacity of 512MB and can be partitioned for storage based on the multiple cache layers 130. Each entry in the cache directory table includes "cache directory of cache layer - status bit - list of holder processing layers - dirty bit".

[0084] The global cache controller 150 can achieve consistency maintenance. It adopts a directory-based consistency protocol to maintain a "cache directory table" that records the state of each cache unit, such as not cached, shared, exclusive, modified, etc., as well as the processing layer of the holder of each cache unit in the cache layer 130.

[0085] When a write operation is performed at the holder processing layer, the write operation consistency guarantee process includes the following:

[0086] (1) Exclusive request initiation and cache directory query.

[0087] When the holder processing layer performs a write operation, if the L3 cache of the holder processing layer is missed, it sends an "exclusive write request" to the cache layer 130 through the first interface controller 122. The request packet contains an 8-bit holder processor address, a 64-bit target address block, and 512 bits of write data.

[0088] Based on the records of the processor addresses that hold all cache units, cache layer 130, upon receiving a request, queries its local cache directory table to obtain the current status and list of holders of the cache unit. The local cache directory table is stored in a distributed manner, with each cache layer 130 maintaining its own local address block directory.

[0089] If the status of cache layer 130 is "not cached": directly allocate space to cache layer 130 to store data, update the status bit of the cache directory table to "exclusive", record the holder's processing layer as the current processing layer, i.e. the holder's processing layer, and return a "write authorization" response to the holder's processing layer.

[0090] If the status of cache layer 130 is "shared" or "modified": initiate the consistency maintenance process.

[0091] (2) Issuance and confirmation of invalidation command.

[0092] The global cache controller 150 can generate an "invalidation command" based on the holder list in the cache directory table. The invalidation command includes the cache layer ABID and the invalidation identifier. It is sent to all holder processing layers through the highest QoS priority consistency message channel of the interconnection layer 140.

[0093] For cache layer 130 that is frequently shared across multiple processing layers, for example, if the number of times it is shared in the cache directory table is greater than 10 times / ms, an invalidation command is pre-sent to the holder through interconnect layer 140 one clock cycle before receiving an exclusive request to reduce waiting latency. The pre-invalidation command is marked as "pending confirmation" and only takes effect when an exclusive request is received subsequently.

[0094] After receiving the invalidation command, the holder processing layer invalidates the corresponding blocks in the local L1 cache, L2 cache, and L3 cache. It does not need to return an acknowledgment immediately, but instead attaches the acknowledgment information when initiating a new request. The delayed acknowledgment window can be less than or equal to 5ns, reducing the number of consistency message interactions and lowering overhead.

[0095] (3) Granting write permissions and updating the directory.

[0096] The global cache controller 150 can wait for all holders to return acknowledgment responses. The timeout threshold for returning acknowledgment responses is 10ns; if the timeout occurs, the invalidation command is resent. After receiving acknowledgment responses from all holders, a "write authorization" response is sent to the processing layer that initiated the write operation. The cache layer 130 updates the cache directory table. The state of the cache layer 130 is updated to "modified," recording that the current holder is the write operation processing layer, marking a dirty bit (1 bit), and updating the access timestamp for subsequent replacement algorithm judgment.

[0097] The process for ensuring data consistency across multiple processing layers includes the following:

[0098] (1) Loading shared data.

[0099] The first processing layer initiates a data read request. The cache layer 130 queries the cache directory table and finds the status to be "not cached." It then retrieves data from external DDR5 memory (Double Data Rate 5 Random Access Memory), for example, through the DDR memory controller 210 of the first processing layer (see reference). Figure 4The cache layer loads data into the cache unit, updates the directory status to "shared," records the list of holders for the current processing layer, and returns the data to the current processing layer. When other processing layers request the same data subsequently, the cache layer 130 directly reads the data from the cache unit and returns it, while updating the list of holders for the newly added processing layer. The directory status remains "shared," eliminating the need to reload memory data and reducing memory access latency.

[0100] (2) Handling write conflicts for shared data.

[0101] When any shared processing layer initiates a write request, the write operation consistency guarantee process is triggered, and the global cache controller 150 sends an invalidation command to all other holders. After other holders complete the invalidation and return confirmation, the write request processing layer obtains exclusive write permission. At this time, the holder list in the directory table only retains this processing layer, and its status is updated to "modified". If other processing layers subsequently request the data again, the cache layer 130 must first trigger a write-back operation to write the cache layer data in the "modified" state back to memory, then load the data into the request processing layer cache, and simultaneously update the directory status to "shared", adding the new processing layer to the holder list.

[0102] Distributed directory synchronization mechanisms include the following:

[0103] (1) Directory storage and retrieval.

[0104] The global directory is divided according to the cache directory of cache layer 130. Each cache layer 130 maintains a directory table fragment (including status, holder, and dirty bit) of the local management address block to avoid single-point directory bottlenecks. When the processing layer accesses the cache directory of a non-local cache layer, the global cache controller 150 uses an atomic operation synchronization mechanism to implement "read-modify-write" atomic operations in hardware with a latency of less than or equal to 5ns. It then sends a directory query request to the target cache layer. After the target cache layer returns the directory information, subsequent consistency operations are performed.

[0105] (2) Directory update synchronization.

[0106] Directory update synchronization is ensured through atomic operations. For cross-core directory updates, such as adding or deleting processing layer addresses in a shared list, atomic operation circuitry is used to ensure the atomicity of the update, avoiding directory inconsistencies caused by concurrent operations across multiple processing layers. After the directory update is complete, the target cache layer sends a "synchronization complete" notification to the global cache controller 150. The cache controller records the update timestamp for subsequent directory consistency verification during fault recovery.

[0107] The third processor microcore M2 is used to respond to a replacement request for multiple candidate cache units received from multiple cache layers 130, select a candidate cache unit with a priority lower than a preset standard as the cache unit to be replaced, wherein the priority of the cache unit is calculated based on the access frequency and access time interval of the cache unit; and replace the data in the cache unit to be replaced.

[0108] The directory synchronization accelerator 1533 is an atomic operation hardware circuit used to perform atomic updates to the address mapping table after migrating data from the target cache layer to an available cache layer 130 among multiple cache layers 130. The atomic operation hardware circuit supports atomic "read-write-modify" operations on the target cache layer with a synchronization latency of less than or equal to 5ns, avoiding the overhead of software synchronization.

[0109] The priority calculation accelerator 1532 is used to calculate the priority of each cache unit in each processing layer 120 based on the access frequency and access time interval of the cache units. The priority calculation accelerator may include fixed-point arithmetic circuitry. The fixed-point arithmetic circuitry can calculate the priority score P of each cache unit in real time. The formula for calculating the priority score P is:

[0110] P=α×F+β×T+γ×I (2);

[0111] Where F is a normalized access frequency counter, which increments after each access and decays periodically; T is the temporal locality factor, calculated based on the reciprocal of the most recent access time interval; I is the application importance weight, predefined by the system or user for different tasks or address spaces; α, β, and γ are adjustable weight coefficients, initially set by the system and dynamically adjusted according to the load mode during runtime. It can be dynamically configured in steps of 0.1, ranging from 0 to 1, and the computational latency can be controlled within two clock cycles.

[0112] The global cache controller 150 can implement the cache replacement algorithm through the third processor microcore M2. By supporting dynamic adjustment of weight coefficients, it can eliminate the lowest priority block in the world during replacement, taking into account both the principle of locality and global load balancing.

[0113] A globally priority-aware adaptive replacement algorithm is adopted, which combines the principle of locality with the global load status. Based on factors such as access frequency, access time and application importance, a priority score is dynamically calculated for each cache unit, and the cache unit with the lowest global priority is replaced when replacement is needed.

[0114] The cache replacement algorithm includes a priority quantization model, an adaptive strategy adjustment method, and a distributed collaborative replacement process.

[0115] The priority quantization model maintains a dynamic priority score P for each cache unit.

[0116] The adaptive strategy adjustment method periodically monitors the load rate of each cache layer 130, which includes cache space occupancy and access pressure. When the load rate of cache layer 130 remains above a threshold, the priority threshold for replacement operations in cache layer 130 is automatically increased, i.e., the replacement threshold is raised. Simultaneously, the weight coefficient β of temporal locality is appropriately reduced to delay the replacement of frequently accessed cold data. When a certain type of application exhibits strong spatial locality, such as consecutive address access, the weight of the T factor in cache layer 130 for that type of data is temporarily increased to reduce its probability of replacement.

[0117] The distributed collaborative replacement process includes: when any cache layer 130 needs to replace a cache unit, the local controller of cache layer 130 can send a replacement request to the global cache controller 150. The global cache controller 150 aggregates the priority scores of candidate cache units in all cache layers 130 and selects the cache unit with the lowest global priority score as the replacement cache unit. If the selected replacement cache unit (victim, the object to be evicted) contains dirty data, a write-back operation is triggered through the interconnection layer 140, and the relevant processing layers are notified to update their cache directory status.

[0118] Priority score caching can store the priority scores of the most recent 100,000 cache blocks, with each block having a 16-bit priority score. A write-back strategy can be employed to reduce frequent access to cache units. A cache block is a cache unit. Non-volatile storage can include 16MB of BeFuse (Buried Fuse) storage, which can be used to persist configuration parameters such as the initial mapping table, default weight coefficients, and fault thresholds. These parameters are automatically loaded upon system power-on to prevent configuration loss.

[0119] The fourth processor microcore M3 is used to respond to the received fault reporting signal of cache layer 130, determine that the cache layer 130 that sent the fault reporting signal is unavailable, prohibit the forwarding of new access requests to the unavailable cache layer 130, and migrate the data in the unavailable cache layer 130 to the available cache layer 130 among the multiple cache layers 130.

[0120] The global cache controller 150 can handle faults. By detecting a fault, it quickly migrates address blocks and prohibits access by new requests, ensuring system continuity.

[0121] (1) Fault handling process.

[0122] When cache layer 130 detects an uncorrectable ECC error or a power failure, it sends a fault reporting signal to the global cache controller 150. The fault reporting signal includes the fault type and the cache directory of cache layer 130. The faulty cache layer is immediately marked as "unavailable," preventing new requests from being forwarded to it. All processing layers are notified via interconnect layer 140 to suspend access to the faulty cache layer. A dynamic remapping mechanism is initiated to migrate the address blocks of the faulty cache layer to the normal cache layer. After the migration is complete, the access suspension is lifted without affecting system continuity.

[0123] (2) Performance monitoring implementation.

[0124] The global cache controller 150 may also include an internal monitoring interface 155. The internal monitoring interface 155 includes a JTAG (Joint Test Action Group) debugging interface and performance counters. The JTAG debugging interface supports real-time reading of address mapping tables, cache directory tables, and performance counter data, supporting system debugging and maintenance.

[0125] The performance counters can integrate 16 32-bit counters to respectively count the cache hit rate of the global cache layer or a single cache layer, the number of address remappings, the consistency message latency, and the number of replacement operations, with a statistical period of 100ns. A performance monitoring report can be generated every 1ms, including the average, maximum, and minimum values ​​of each metric. This report is uploaded to the system management software via the PCIe management interface, allowing users to view it in real time and optimize performance.

[0126] The Global Cache Controller 150 can perform four functions: address mapping management, consistency maintenance, cache replacement algorithm execution, and fault handling.

[0127] The first processor microcore M0, the second processor microcore M1, the third processor microcore M2, and the fourth processor microcore M3 each include their own on-chip bus interface G. The first processor microcore M0, the second processor microcore M1, the third processor microcore M2, and the fourth processor microcore M3 transmit data to each other through the on-chip bus, enabling nanosecond-level data synchronization.

[0128] Figure 6 A schematic diagram of a cache layer according to an embodiment of this application is shown.

[0129] like Figure 1 and Figure 6As shown, cache layer 130 includes multiple cache units 131 and a second interface controller 132. Data is stored in the cache units 131. Cache layer 130 also includes a switch array 133, through which the multiple cache units 131 are interconnected. The multiple cache units 131 can be accessed in parallel with the switch array 133 through a block layout, which can reduce access latency. The switch array 133 can be a cross-connected switch array. A unified cross-core cache layer 130 is constructed to achieve global sharing and management of cache resources.

[0130] The second interface controller 132 is connected to multiple cache units 131 and is connected to the global cache controller 150 through a vertical connection channel. The second interface controller 132 is configured to send read and write instructions to the corresponding cache unit 131 among the multiple cache units 131 based on the access operation of the global cache controller 150, and return the response data generated by the cache unit 131 based on the read and write instructions to the global cache controller 150.

[0131] Vertical connection channels can be vertical electrical connection links constructed using TSV technology. For example... Figure 1 and Figure 5 As shown, the cache layer 130 and the global cache controller 150 can establish a connection via a TSV link to achieve direct interconnection between layers. The vertical connection channel adopts a custom high-speed protocol and supports address remapping instruction issuance, directory synchronization requests, and fault status reporting via the TSV link.

[0132] The second interface controller 132 can process access requests and consistency instructions from the interconnect layer 140 and the global cache directory, and issue read and write instructions to the cache unit 131. The second interface controller 132 can also return response data to the interconnect layer 140 and report status information to the global cache directory.

[0133] The second interface controller 132 can be referred to as the L4 interface controller. The second interface controller 132 may include a request processing queue 1321, a response generation unit 1324, a global directory interaction interface 1322, and a performance monitoring unit 1323.

[0134] The request processing queue 1321 can be used to receive and temporarily store external requests sequentially, and then push the external requests to the cache unit 131. The queue depth of the request processing queue 1321 is 32. The response generation unit 1324 can be used to receive response data from the cache unit 131, encapsulate the response data from the cache unit 131 into standardized data packets conforming to the CXL protocol, and then send them to the interconnect layer 140. The global directory interaction interface 1322 can be used to communicate bidirectionally with the global cache controller 150, both reporting the status of the storage blocks of the cache layer 130 and receiving and processing consistency instructions. The performance monitoring unit 1323 can be used to continuously collect access data of the cache unit 131, such as hit rate and latency, and report the summarized performance indicators to the second interface controller 132. The second interface controller 132 may also include an address mapping management unit.

[0135] The second interface controller 132 can implement address mapping and parse the CXL heterogeneous cache consistency protocol. It supports out-of-order processing and in-order response by integrating a request processing queue 1321 with a depth of 32. At the same time, it interfaces with the global directory to maintain cache consistency. It forms an efficient data path between the L4 interface controller, cache controller, and SRAM storage array through read and write circuits.

[0136] The cache unit 131 may include a storage array 1311 and a cache controller 1313.

[0137] The storage array 1311 includes multiple storage banks arranged in an array. The storage array 1311 receives read and write instructions, performs read and write operations based on these instructions, and returns the response data generated by the read and write operations to the cache controller 1313. The storage banks can be on-chip SRAM (Static Random-Access Memory) caches. The storage array 1311 can be an integrated 512MB 16-way set-associative SRAM storage array, employing a distributed storage architecture that supports parallel access to multiple storage banks. The storage array 1311 also includes a controller B, which can receive storage bank-level access instructions from the cache controller 1313 and send specific read and write instructions to the target storage bank. At the physical layout level, the storage array 1311 can adopt a multi-module block structure, such as an 8×8 array, with full interconnection between modules achieved through a CrossBar array.

[0138] The cache controller 1313 is used to receive read / write commands from the second interface controller 132, issue read / write commands to the storage array 1311, and return response data from the storage array 1311 to the second interface controller 132. The response data from the storage array 1311 may include request data and operation status.

[0139] The cache controller 1313 may include a tag comparison unit, a cache replacement unit, and an ECC verification unit. The tag comparison unit is used to match address information and return a hit or miss result to the cache controller 1313. The cache replacement unit is used to calculate the address of the memory block to be replaced upon receiving a replacement request. The ECC verification unit is used to perform real-time verification of read and write instructions and can output various operation statuses such as "normal," "error," and "corrected data." The cache controller 1313 can implement three main functions: tag comparison, replacement strategy, and ECC error detection and correction.

[0140] The cache unit 131 also includes a read / write optimization circuit 1312. The read / write optimization circuit 1312 is connected between the storage array 1311 and the cache controller 1313. The read / write optimization circuit 1312 is used to amplify the signal carrying the response data from the storage array 1311 and transmit it to the cache controller 1313, and to drive the storage array 1311 to perform read / write operations according to the read / write instructions from the cache controller 1313.

[0141] The read / write optimization circuit 1312 can be designed for low-voltage environments, integrating a high-performance sensitive amplifier and word line driver circuit, and can operate stably at low voltages, such as 1.2V. The sensitive amplifier can detect and amplify weak signals from the memory array 1311, transmitting the amplified, clear signal from the memory array 1311 to the cache controller 1313. The word line driver circuit can receive read / write commands from the cache controller 1313, such as word line control signals, and drive the word lines in the memory array 1311. The sensitive amplifier and word line driver circuit work together to achieve overall signal optimization in response to data and read / write commands.

[0142] By optimizing the circuit and logic designs of the processing layer 120, cache layer 130, interconnect layer 140, and global cache controller 150 at the circuit level, low-latency, high-bandwidth interaction can be achieved. In the circuit design of the processing layer 120, the interface logic between the L3 and L4 caches can be optimized to ensure efficient data interaction between the local and global caches. In the circuit design of the interconnect layer 140, the design of the serial-to-parallel / parallel-to-serial converter (SerDes) and the routing controller can be optimized. In the circuit design of the cache layer 130, the layout and read / write circuitry of the SRAM memory array can be optimized to reduce cache access latency, while the cache controller and L4 interface controller are designed. In the circuit design of the global cache controller 150, the parsing of the CXL protocol and the maintenance of global cache consistency can be optimized.

[0143] This application also provides a data access method, which can be executed by the encapsulation structure of the embodiments of this application.

[0144] Figure 7 A flowchart illustrating the operation of a data access method according to an embodiment of this application is shown.

[0145] like Figure 7 As shown, the data access method includes operations S710 to S740.

[0146] When operating the S710, the processing layer generates an access request.

[0147] When operating the S720, the interconnect layer selects a target transmission path from multiple transmission paths between the processing layer and the global cache layer based on the access request from the processing layer, and provides the access request to the global cache controller through the target transmission path.

[0148] When operating the S730, the global cache controller performs access operations on the cache layer based on the access requests provided by the interconnect layer, obtains the response data generated based on the access operations from the cache layer, and provides the response data to the interconnect layer.

[0149] When operating the S740, the interconnect layer returns the response data provided by the global cache controller to the processing layer through the target transmission path.

[0150] According to an embodiment of this application, the data access method further includes: before performing an access operation on a cache layer according to an access request provided by an interconnect layer, the global cache controller divides the global address space for multiple cache layers into multiple address blocks, each address block corresponding to a cache unit in the cache layer; maintains a cache directory for each cache layer among the multiple cache layers, the cache directory recording the holder of the cache unit, the holder including at least one processing layer among multiple processing layers; wherein performing an access operation on a cache layer according to an access request provided by an interconnect layer includes: in response to receiving an access request through the interconnect layer, determining the target address block corresponding to the access request from the multiple address blocks; determining the target cache layer where the target address block is located from the multiple cache layers by querying an address mapping table; determining the holder of the cache unit corresponding to the target address block according to the cache directory of the target cache layer, and sending an invalidation command to the holder through the interconnect layer, the invalidation command being used to invalidate the access rights of the holder's processing layer to the cache unit corresponding to the target address block; if the target cache layer is available, forwarding the access request to the interconnect layer; if the target cache layer is unavailable, migrating the data in the target cache layer to an available cache layer among the multiple cache layers, and sending a notification for updating the address mapping table to the multiple processing layers through the interconnect layer.

[0151] According to an embodiment of this application, the data access method further includes: a global cache controller responding to receiving a replacement request for multiple candidate cache units from multiple cache layers, selecting a candidate cache unit with a priority lower than a preset standard as the cache unit to be replaced, wherein the priority of the cache unit is calculated based on the access frequency and access time interval of the cache unit; and the global cache controller performing data replacement on the data in the cache unit to be replaced.

[0152] According to an embodiment of this application, the data access method further includes: in response to receiving a fault reporting signal from a cache layer, the global cache controller determines that the cache layer that sent the fault reporting signal is unavailable; the global cache controller prohibits forwarding new access requests to the unavailable cache layer; and the global cache controller migrates data in the unavailable cache layer to an available cache layer among multiple cache layers.

[0153] According to an embodiment of this application, the data access method further includes: after receiving an access request from the interconnect layer, the global cache controller parses the access request based on the target cache consistency protocol; and before providing the response data to the interconnect layer, the global cache controller packages the response data based on the target cache consistency protocol.

[0154] Figure 8 A schematic diagram of a data access method according to an embodiment of this application is shown.

[0155] like Figure 8 As shown, the processing layer can be a processor chip. The interconnect layer can be an interconnect chip. The cache layer can be an L4 cache chip.

[0156] Processor chip access to L4 cache includes (1) local cache lookup phase, (2) interconnect chip forwarding phase and (3) L4 cache chip response phase.

[0157] S1. The processor core sequentially queries the L1, L2 and L3 caches. If none of them are hit, the L3 cache forwards the request to the L3-L4 interface controller, which then encapsulates the request packet.

[0158] S2. The processor chip's local interconnect interface sends the request packet to the interconnect chip via 2-4 TSV links. Operation S2 includes the following operations:

[0159] 1. The routing controller parses the ID in the request and determines the output port.

[0160] 2. The traffic scheduler (or route controller) establishes a data transmission path based on the Weighted Fair Queuing (WFQ) algorithm and forwards request packets to the output FIFO in the direction of the L4 cache core.

[0161] 3. Output FIFO (First-In-First-Out, i.e., the input port DR of the flow scheduler) to reassemble data, and the protocol adaptation unit adds a link layer header and a new CRC checksum.

[0162] 4. After the PHY layer (i.e., the physical layer interface PHY) performs parallel-to-serial conversion.

[0163] 5. The SerDes module (i.e., SerDes) sends data to the target L4 cache kernel via 4-8 links.

[0164] S3. The interconnect chip sends a request packet to the global cache controller.

[0165] S4. The global cache controller receives the request packet, extracts the target address and operation type, and sends the request to the target L4 cache core.

[0166] The S5 L4 cache core performs the following operations:

[0167] 1. The L4 interface controller receives request packets from the global cache controller and checks the local cache directory table to confirm the status of cache blocks (i.e., cache units). Cache block status includes uncached, shared, exclusive, and modified.

[0168] 2. The cache controller decomposes the target address into a 4-bit block address, a 10-bit set address, and a 6-bit block address, triggering parallel access to the SRAM memory array. If it is a read operation, the cache controller reads the data from the SRAM memory array, encapsulates the data and response identifiers (success / error) into a CXL protocol response packet, and sends it to the L4 interface controller.

[0169] 3. The L4 interface controller sends the response packet to the interconnect chip.

[0170] S6. The interconnect chip returns the response packet along its original path to the L3-L4 interface controller of the processor chip.

[0171] S7. The processor chip performs the following operations:

[0172] After verifying the integrity of the response packet, the L3-L4 interface controller updates the L3 cache data and returns the data to the computing core (i.e., computing core C), completing the entire access process.

[0173] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0174] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A package structure, characterized by, The packaging structure comprises a substrate, a processing layer, a cache layer, an interconnection layer and a global cache controller, wherein: The interconnection layer and the global cache controller are located on the substrate and are connected to each other through physical channels on the substrate; The processing layer is located on the side of the interconnection layer away from the substrate and is connected to the interconnection layer; The cache layer is located on the side of the global cache controller away from the substrate and is connected to the global cache controller; The global cache controller is configured to perform access operations on the cache layer according to access requests provided by the processing layer, and return response data generated by the cache layer based on the access operations to the processing layer; The interconnection layer is configured to route data transmission between the processing layer and the global cache controller; The interconnection layer comprises a plurality of first ports and a plurality of second ports, the plurality of first ports being connected to the processing layer, and the plurality of second ports being connected to the global cache controller; The global cache controller comprises a plurality of third ports, and the plurality of third ports are connected to the plurality of second ports of the interconnection layer through a plurality of physical channels on the substrate; The cache layer comprises a plurality of cache units, and the cache units store data, and the global cache controller further comprises: A multi-core control cluster connected to the plurality of third ports, configured to receive access requests from the interconnection layer through the plurality of third ports, determine the cache units to be accessed by querying an address mapping table according to the access requests, perform access operations on the cache units to be accessed, obtain response data generated by the cache units to be accessed based on the access operations, and provide the response data to the interconnection layer; A hardware acceleration module connected to the multi-core control cluster, configured to provide hardware acceleration for address mapping table queries of the multi-core control cluster; A storage unit connected to the multi-core control cluster, configured to store the address mapping table.

2. The package structure of claim 1, wherein, The interconnection layer further comprises a routing controller, and there are a plurality of transmission paths between the plurality of first ports and the plurality of second ports, wherein: The routing controller is configured to, in response to a target first port in the plurality of first ports receiving an access request from the processing layer, select a target second port from the plurality of second ports according to the access request, determine a target transmission path between the target first port and the target second port in the plurality of transmission paths, provide the access request to the global cache controller through the target transmission path, and return response data provided by the global cache controller for the access request to the processing layer through the target transmission path.

3. The package structure of claim 2, wherein, The interconnection layer further comprises a protocol adaptation unit configured to convert the access request into a data format conforming to a target cache coherence protocol before providing the access request to the global cache controller through the target transmission path.

4. The package structure of claim 2, wherein, The number of second ports is greater than the number of first ports.

5. The package structure of claim 2, wherein, The access requests are divided into a plurality of types, different types of access requests have corresponding priorities, and the routing controller is configured to determine the target transmission path between the target first port and the target second port in the plurality of transmission paths by allocating crossbar resources according to the priorities of the access requests based on a weighted fair queue scheduling algorithm.

6. The package structure of claim 2, wherein, The processing layer comprises a plurality of local interconnection interfaces, and the plurality of local interconnection interfaces are connected to the plurality of first ports of the interconnection layer through a plurality of vertical connection channels.

7. The package structure of claim 6, wherein, The processing layer further includes a plurality of computing cores, a first level cache, a second level cache, and a third level cache, and the cache layer is a globally shared cache.

8. The package structure of claim 7, wherein, The processing layer further includes a first interface controller connected between the third level cache and the local interconnection interface, and the first interface controller is configured to, in response to a query that none of the first level cache, the second level cache, and the third level cache is hit, send, through at least one of the plurality of local interconnection interfaces, an access request for the globally shared cache to the interconnection layer.

9. The package structure of claim 2, wherein, The third port and the second port are serial interfaces supporting a target cache coherence protocol.

10. The package structure of claim 1, wherein, The multi-core control cluster is further configured to, after receiving the access request, parse the access request based on the target cache coherence protocol, and before providing the response data to the interconnection layer, package the response data based on the target cache coherence protocol.

11. The package structure of claim 10, wherein, The number of cache layers and the number of processing layers are both plural; The multi-core control cluster includes: The first processor microkernel is configured to divide a global address space for the plurality of cache layers into a plurality of address blocks, the address blocks corresponding to cache units; in response to receiving the access request through the interconnection layer, determine a target address block corresponding to the access request from the plurality of address blocks, and determine a target cache layer in which the target address block is located from the plurality of cache layers by querying an address mapping table; if the target cache layer is available, forward the access request to the interconnection layer, and if the target cache layer is unavailable, migrate data in the target cache layer to an available cache layer from the plurality of cache layers, and send a notification for updating the address mapping table to the plurality of processing layers through the interconnection layer; The second processor microkernel is configured to maintain a cache directory for each cache layer from the plurality of cache layers, the cache directory recording a holder of each cache unit in the cache layer, the holder including at least one processing layer from the plurality of processing layers; determine a holder of a cache unit corresponding to the target address block according to the cache directory of the target cache layer, and send an invalidation command to the holder through the interconnection layer, the invalidation command being used to invalidate access rights of the holder processing layer to the cache unit corresponding to the target address block.

12. The package structure of claim 11, wherein, The multi-core control cluster further includes a third processor microkernel configured to, in response to receiving replacement requests for a plurality of candidate cache units from the plurality of cache layers, select a candidate cache unit with a priority lower than a preset standard from the plurality of candidate cache units as a cache unit to be replaced, wherein the priority of the cache unit is calculated according to an access frequency and an access time interval of the cache unit; and perform data replacement on data in the cache unit to be replaced.

13. The package structure of claim 12, wherein, The multi-core control cluster further includes a fourth processor microkernel configured to, in response to receiving a failure reporting signal of a cache layer, determine the cache layer sending the failure reporting signal as unavailable, prohibit forwarding of a new access request to the unavailable cache layer, and migrate data in the unavailable cache layer to an available cache layer from the plurality of cache layers.

14. The package structure of claim 13, wherein, The first processor microkernel, the second processor microkernel, the third processor microkernel, and the fourth processor microkernel perform data transmission through an on-chip bus.

15. The package structure of claim 11, wherein, The hardware acceleration module includes at least one of the following: An operation accelerator is configured to generate an index of an address block by performing a hash calculation on identification information of the address block, the index of the address block being used for a query of the address mapping table; A priority calculation accelerator is configured to calculate a priority of each cache unit of each processing layer according to a frequency of access and an interval of access of the cache unit; A directory synchronization accelerator is an atomic operation hardware circuit configured to perform an atomic update on the address mapping table after migrating data in a target cache layer to a cache layer available among the plurality of cache layers.

16. The package structure of claim 2, wherein, The cache layer includes: a plurality of cache units in which data is stored, the plurality of cache units being connected to each other through a switch array; a second interface controller connected to the plurality of cache units and connected to the global cache controller through a vertical connection channel, the second interface controller being configured to send a read-write instruction to a corresponding cache unit of the plurality of cache units based on an access operation of the global cache controller, and return response data generated by the cache unit based on the read-write instruction to the global cache controller.

17. The package structure of claim 16, wherein, The cache unit includes a storage array and a cache controller, wherein: the storage array includes a plurality of storage blocks arranged in an array, the storage array being configured to receive a read-write instruction, perform a read-write operation based on the read-write instruction, and return response data generated based on the read-write operation to the cache controller; the cache controller is configured to receive the read-write instruction from the second interface controller, issue the read-write instruction to the storage array, and return the response data of the storage array to the second interface controller.

18. The package structure of claim 17, wherein, The cache unit further includes a read-write optimization circuit connected between the storage array and the cache controller, the read-write optimization circuit being configured to amplify and transmit a signal carrying the response data from the storage array to the cache controller, and drive the storage array to perform a read-write operation according to the read-write instruction from the cache controller.

19. A data access method, executed by the package structure of any one of claims 1 to 18, characterized in that, The data access method includes: a processing layer generating an access request; an interconnection layer selecting a target transmission path among a plurality of transmission paths between the processing layer and a global cache layer according to the access request from the processing layer, and providing the access request to the global cache controller through the target transmission path; the global cache controller performing an access operation on the cache layer according to the access request provided by the interconnection layer, obtaining response data generated based on the access operation from the cache layer, and providing the response data to the interconnection layer; the interconnection layer returning the response data provided by the global cache controller to the processing layer through the target transmission path.

20. The data access method of claim 19, wherein, The data access method further includes: the global cache controller dividing a global address space for the plurality of cache layers into a plurality of address blocks before performing the access operation on the cache layer according to the access request provided by the interconnection layer, the address blocks corresponding to cache units in the cache layer; maintaining a cache directory for each cache layer among the plurality of cache layers, the cache directory recording a holder of the cache unit, the holder including at least one processing layer among the plurality of processing layers; wherein performing the access operation on the cache layer according to the access request provided by the interconnection layer includes: determining a target address block corresponding to the access request from the plurality of address blocks in response to receiving the access request through the interconnection layer; determining a target cache layer in which the target address block is located from the plurality of cache layers by querying the address mapping table; determining a holder of the cache unit corresponding to the target address block according to a cache directory of the target cache layer, and sending an invalidation command to the holder through the interconnection layer, the invalidation command being used for invalidating, by the holder processing layer, access authority of the cache unit corresponding to the target address block; forwarding the access request to the interconnection layer in a case where the target cache layer is available; migrating data in the target cache layer to a cache layer available in the plurality of cache layers in a case where the target cache layer is unavailable, and sending a notification for updating an address mapping table to the plurality of processing layers through the interconnection layer.

21. The data access method of claim 20, wherein, The data access method further comprises: selecting, by the global cache controller, a candidate cache unit with a priority lower than a preset standard from the plurality of candidate cache units as a cache unit to be replaced in response to receiving replacement requests for the plurality of candidate cache units from the plurality of cache layers, wherein the priority of the cache unit is calculated according to an access frequency and an access time interval of the cache unit; performing data replacement on data in the cache unit to be replaced by the global cache controller.

22. The data access method of claim 21, wherein, The data access method further comprises: determining, by the global cache controller, a cache layer sending the failure reporting signal as unavailable in response to receiving the failure reporting signal of the cache layer; prohibiting, by the global cache controller, forwarding of a new access request to the unavailable cache layer; and migrating, by the global cache controller, data in the unavailable cache layer to a cache layer available in the plurality of cache layers.

23. The data access method of claim 19, wherein, The data access method further comprises: analyzing, by the global cache controller, the access request based on a target cache coherence protocol after receiving the access request from the interconnection layer; and packing, by the global cache controller, the response data based on the target cache coherence protocol before providing the response data to the interconnection layer.

Images