Computing device and data access method

Abstract

The invention provides a computing device and a data access method. The device comprises a plurality of computing units; the plurality of three-dimensional stacked memory devices are respectively in one-to-one correspondence with the plurality of computing units, the plurality of three-dimensional stacked memory devices adopt unified continuous addressing on a logic address space, and each three-dimensional stacked memory device is allocated with a unique and continuous address space; the data caching module is respectively connected with each computing unit and each three-dimensional stacked storage device through a crossbar switch, and is used for receiving a data access request sent by any computing unit, analyzing a logic address in the data access request, determining a target three-dimensional stacked storage device corresponding to the logic address, and sending the target three-dimensional stacked storage device to the computing unit; and copying the target data from the target three-dimensional stacked memory device and returning the target data to the computing unit. According to the method and the device, data transfer and synchronization processes of an intermediate computing unit are not needed, so that the data access process is simplified, the delay of data transmission and bandwidth occupation are reduced, and the data access efficiency and the overall performance of the system are improved.

Description

Technical Field

[0001] The embodiments of this application relate to, but are not limited to, the field of computer technology, and in particular to a computing device and a data access method. Background Technology

[0002] Currently, in computing architectures containing multiple computing units, each computing unit is equipped with an independent main memory (such as a three-dimensional stacked storage device) to store its data. When a computing unit requests access to data in the three-dimensional stacked storage device corresponding to another computing unit, it must first send an access request to that other computing unit. The other computing unit then retrieves the data from its corresponding three-dimensional stacked storage device, transfers the data to a shared cache, and then synchronizes the data with the requesting computing unit. The requesting computing unit then loads the data from the shared cache into its local cache. In this process, data undergoes multiple transfers, making the process quite complex. Summary of the Invention

[0003] This application aims to solve one of the technical problems in the related art to a certain extent, and provides a computing device, chip product, computer equipment and data access method. The technical solution provided by this application includes the following aspects.

[0004] A first aspect of this application provides a computing device, comprising:

[0005] Multiple computing units;

[0006] Multiple three-dimensional stacked storage devices are respectively associated with multiple computing units. The multiple three-dimensional stacked storage devices adopt a unified continuous addressing in the logical address space, and each of the three-dimensional stacked storage devices is allocated a unique and continuous address space.

[0007] The data caching module is connected to each of the computing units and each of the three-dimensional stacked storage devices via cross switches. It is used to receive data access requests sent by any of the computing units, parse the logical address in the data access request, determine the target three-dimensional stacked storage device corresponding to the logical address according to the unified continuous addressing, copy the target data from the target three-dimensional stacked storage device and return it to the computing unit.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the data caching module includes:

[0009] Multiple cache slices are connected one-to-one with the multiple three-dimensional stacked storage devices;

[0010] Each cache slice includes an independent static random access memory, a first interface, and a second interface. The first interface is connected to each computing unit via a crossbar switch, and the second interface is connected to the corresponding three-dimensional stacked storage device.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, each cache slice further includes:

[0012] The address resolution circuit is used to resolve the logical address in the data access request and determine the target three-dimensional stacked storage device corresponding to the logical address.

[0013] A hit detection circuit is used to detect whether the target data pointed to by the data access request has been stored in the static random access memory;

[0014] A data transfer circuit is used to copy the target data from the target three-dimensional stacked storage device to the static random access memory when the hit detection circuit detects that the target data is not stored in the static random access memory.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the data caching module further includes:

[0016] An arbitration circuit is configured to merge the requests from multiple computing units when multiple computing units simultaneously send data access requests for the same data block in a target three-dimensional stacked storage device, and then broadcast the same data block to the multiple computing units at once.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the three-dimensional stacked storage device is a three-dimensional stacked dynamic random access memory.

[0018] A second aspect of this application provides a data access method applied to the aforementioned computing device, comprising:

[0019] Receive a data access request sent by the computing unit, wherein the data access request carries the logical address corresponding to the target data;

[0020] Based on the logical address, determine whether the target data has been stored in the data cache module;

[0021] If the target data is already stored in the data cache module, then the target data is read from the data cache module and returned to the computing unit;

[0022] If the target data is not stored in the data cache module, the corresponding target three-dimensional stacked storage device is determined according to the logical address, the target data is obtained from the target three-dimensional stacked storage device through the data cache module, and returned to the computing unit.

[0023] In conjunction with the second aspect, in certain implementations of the second aspect, the process of determining the corresponding target three-dimensional stacked storage device based on the logical address includes:

[0024] Based on the unified continuous addressing of multiple three-dimensional stacked storage devices in the logical address space, the logical address range to which the logical address belongs is determined, and the three-dimensional stacked storage device corresponding to the logical address range is taken as the target three-dimensional stacked storage device.

[0025] In conjunction with the second aspect, in some implementations of the second aspect, the process of obtaining target data from the target three-dimensional stacked storage device through the data caching module includes:

[0026] Data read commands are sent to the target three-dimensional stacked storage device via a cache slice connected to the target three-dimensional stacked storage device.

[0027] Receive the target data returned by the target three-dimensional stacked storage device, and store the target data in the static random access memory of the cache slice;

[0028] The crossbar switch is controlled to connect the data transmission channel between the cache slice and the computing unit, and the target data is returned to the computing unit through the data transmission channel.

[0029] In conjunction with the second aspect, in some implementations of the second aspect, the data access method further includes the following steps:

[0030] If multiple data access requests carrying the same logical address are received simultaneously from multiple computing units, then the multiple data access requests are merged.

[0031] Once the same data block corresponding to the logical address is obtained from the target three-dimensional stacked storage device or the data cache module, the data block is broadcast to the multiple computing units at once.

[0032] In conjunction with the second aspect, in some implementations of the second aspect, the determination of whether the target data has been cached in the data caching module is automatically executed by the hit detection circuit in the data caching module.

[0033] In conjunction with the second aspect, in some implementations of the second aspect, the data access request is a single loading instruction.

[0034] The technical solution provided in this application has the following beneficial effects: The data caching module of this application can realize the cross-interconnection of the multiple computing units and multiple three-dimensional stacked storage devices, enabling the computing units to directly access the non-local corresponding three-dimensional stacked storage devices without the need for data transfer and synchronization processes through intermediate computing units. This design simplifies the data access process, reduces data transmission latency and bandwidth consumption, and improves data access efficiency and overall system performance. Simultaneously, the three-dimensional stacked storage devices adopt unified addressing, with each device allocated a different address space. This addressing method facilitates management and expansion, meeting the needs of large-scale in-memory computing architectures. Attached Figure Description

[0035] Figure 1 A schematic diagram of the architecture of a computing device provided for related technologies;

[0036] Figure 2 A schematic diagram of the architecture of a computing device provided in this application embodiment. Figure 1 ;

[0037] Figure 3 A schematic diagram of the architecture of a computing device provided in this application embodiment. Figure 2 ;

[0038] Figure 4 A flowchart illustrating a data access method provided in an embodiment of this application;

[0039] Figure 5 A schematic diagram of the architecture of a computing device provided in this application embodiment. Figure 3 ;

[0040] Figure 6 This application provides a schematic diagram of the address space partitioning of a three-dimensional stacked storage device cluster.

[0041] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0042] To make the objectives, technical methods, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] It should be noted that although the flowchart shows a logical order, in some cases, the steps shown or described may be performed in a different order than that shown in the flowchart. In the description of the specification, claims, and the foregoing drawings, "at least one" means one or more; "more than" means two or more; "greater than," "less than," and "exceeding" are understood to exclude the stated number; "above," "below," and "within" are understood to include the stated number. The use of terms such as "first," "second," etc., is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly specifying the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0044] In this embodiment, "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: the existence of a alone, the existence of b alone, the existence of c alone, the simultaneous existence of a and b, the simultaneous existence of a and c, the simultaneous existence of b and c, or the simultaneous existence of a, b, and c, where a, b, and c can be single or multiple.

[0045] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0046] Before introducing and explaining the technical solution of this application, some concepts involved in this application will be defined and explained.

[0047] Figure 1 A schematic diagram of the architecture of a computing device provided for related technologies. For example... Figure 1As shown, the architecture of the computing device 10 includes multiple computing units (computing unit 20-1, computing unit 20-2, computing unit 20-3, and computing unit 20-4), each equipped with an independent three-dimensional stacked storage device (three-dimensional stacked storage device 30-1, three-dimensional stacked storage device 30-2, three-dimensional stacked storage device 30-3, and three-dimensional stacked storage device 30-4) to store the data of the corresponding computing unit. The architecture of the computing device 10 also includes a cache module 40 connected to the multiple computing units. The cache module 40 is shared by the multiple computing units and is used to store data frequently accessed by the multiple computing units. When a computing unit requests to access data in the three-dimensional stacked storage device corresponding to another computing unit, it must first send an access request to the other computing unit. The other computing unit then retrieves the data from its corresponding three-dimensional stacked storage device and transfers the data to the shared cache module 40. The cache module 40 then synchronizes the data with the computing unit that requested the data, and the requesting computing unit loads the data from the cache module into its local cache. In the above process, the data needs to undergo multiple transfers, making the process relatively complex.

[0048] In the above technical solution, the data access path generates significant instruction overhead and synchronization latency. Taking the requirement for computing unit 20-1 to read data from the three-dimensional stacked storage device 30-3 as an example, the complete process can be broken down into the following steps:

[0049] First, computing unit 20-1 cannot directly access the three-dimensional stacked storage device 30-3; it requires an explicit collaborative process. This process involves computing unit 20-3 executing a copy instruction to read the target data from the three-dimensional stacked storage device 30-3 and write it to the shared cache module 40.

[0050] Next, all computing units (computing unit 20-1, computing unit 20-2, computing unit 20-3, and computing unit 20-4) must execute an explicit barrier synchronization instruction to ensure that the data write operation of computing unit 20-3 is visible to other computing units. At this synchronization point, all computing units must pause and wait until synchronization is complete.

[0051] Next, the computing unit 20-1 executes a load instruction to read data from the synchronized shared cache module 40 into its local cache or register for subsequent calculations.

[0052] The data flow path described above is from the three-dimensional stacked storage device 30-3 through the computing unit 20-3 to the shared cache module 40, and then after synchronization, it reaches the computing unit 20-1, requiring copy, synchronization, and loading instructions. Among these, the costly remote data transfer and global synchronization operations result in a significant increase in instruction overhead and a decrease in overall bandwidth utilization.

[0053] In the above technical solution, each three-dimensional stacked storage device adopts an independent relative addressing method, for example, the starting address is 0x4000000. The software determines which three-dimensional stacked storage device the requested data originates from, and the shared cache module 40 is managed by software, making the software programming process relatively cumbersome. Because each three-dimensional stacked storage device uses independent relative addressing, the software code needs to explicitly prompt the computing unit 20-3 to request data from the three-dimensional stacked storage device 30-3 and synchronize it to the computing unit 20-1. The purely software-managed cache module 40 needs to reserve a large amount of storage space to ensure versatility, which leads to an increase in chip area.

[0054] This application aims to address one of the technical problems in related technologies to a certain extent by providing a computing device and a data access method.

[0055] The computing device provided in this application includes: multiple computing units; multiple three-dimensional stacked storage devices, each corresponding to one of the multiple computing units, wherein the multiple three-dimensional stacked storage devices adopt a unified contiguous addressing in the logical address space, and each three-dimensional stacked storage device is allocated a unique and contiguous address space; and a data cache module, which is connected to each computing unit and each three-dimensional stacked storage device through a crossbar switch, for receiving data access requests sent by any computing unit, parsing the logical address in the data access request, determining the target three-dimensional stacked storage device corresponding to the logical address according to the unified contiguous addressing, copying target data from the target three-dimensional stacked storage device and returning it to the computing unit.

[0056] Figure 2 A schematic diagram of the architecture of a computing device provided in this application embodiment. Figure 1 .like Figure 2 As shown, the computing device 100 provided in this embodiment includes a computing unit group 200, a three-dimensional stacked storage device group 300, and a data cache module 400. The computing unit group 200 includes multiple computing units (computing unit 200-1, computing unit 200-2, computing unit 200-3, and computing unit 200-4); the three-dimensional stacked storage device group 300 includes multiple three-dimensional stacked storage devices (three-dimensional stacked storage device 300-1, three-dimensional stacked storage device 300-2, three-dimensional stacked storage device 300-3, and three-dimensional stacked storage device 300-4), with each of the multiple three-dimensional stacked storage devices (300-1, 300-2, 300-3, and 300-4) corresponding one-to-one with the multiple computing units (200-1, 200-2, 200-3, and 200-4).

[0057] The three-dimensional stacked storage devices in the three-dimensional stacked storage device group 300 adopt a unified addressing in the logical address space, and each three-dimensional stacked storage device is allocated a unique and contiguous address space; the data cache module 400 is connected to each computing unit and each three-dimensional stacked storage device through cross switches, and is used to receive data access requests sent by any computing unit, parse the logical address in the data access request, determine the target three-dimensional stacked storage device corresponding to the logical address according to the unified contiguous addressing, copy the target data from the target three-dimensional stacked storage device and return it to the computing unit.

[0058] In this embodiment of the application, the computing unit can be a hardware unit with computing capabilities, such as a graphics processing unit (GPU), a tensor processing unit (TPU), a neural processing unit (NPU), a central processing unit (CPU), or a field-programmable gate array (FPGA).

[0059] In this embodiment, the computing unit cluster 200 includes multiple computing units, each capable of collaboratively completing various high-complexity, high-computing-power-requirement specific tasks. Examples include model training, large-scale data mining, high-performance numerical computation, and complex scene simulation.

[0060] In this embodiment, the computing device 100 adopts a one-to-one mapping architecture between computing and storage, meaning that the number of units in the three-dimensional stacked storage device group 300 and the computing unit group 200 are equal. Each three-dimensional stacked storage device is specifically configured to provide efficient, stable, and high-bandwidth-supported large-capacity data storage services to its corresponding computing unit. Specifically, each three-dimensional stacked storage device is responsible for storing various massive working datasets that the associated computing unit needs to process, including but not limited to large-scale training or inference data, complex model parameters, and high-resolution texture resource sets, thereby ensuring that the computing unit can efficiently and with low latency access the required data, improving the overall system's processing performance and resource utilization efficiency.

[0061] In the embodiments of this application, the type of three-dimensional stacked storage device may specifically be dynamic random-access memory (DRAM), three-dimensional dynamic random-access memory (3D-DRAM), high bandwidth memory (HBM), storage-class memory (SCM) / non-volatile memory (NVM), etc.

[0062] For example, the data types stored in the three-dimensional stacked storage device include: training / inference data, such as high-resolution images of sample sets and text sequences in natural language processing; model parameters, such as weight matrices, bias terms, and optimizer states of deep learning models; and intermediate results, such as feature maps output by convolutional layers and key-value pair caches of attention mechanisms.

[0063] In the computing device architecture provided in this application embodiment, the data cache module 400 is responsible for enabling efficient and flexible cross-interconnection between the computing unit group 200 and the three-dimensional stacked storage device group 300. Specifically, when a computing unit requests access to data located in a target three-dimensional stacked storage device associated with another computing unit due to task execution needs, the data cache module 400 dynamically constructs a dedicated data transmission channel, directly connecting the requesting computing unit and the target three-dimensional stacked storage device. Through this channel, the corresponding data stored in the target three-dimensional stacked storage device can be directly transmitted to the requesting computing unit via the data cache module 400. The entire process does not require forwarding or processing by any other computing unit, thus significantly reducing data transmission latency and effectively avoiding the additional overhead and bottlenecks that may be introduced by intermediate nodes. For example, when computing unit 200-1 requests access to main memory data associated with computing unit 200-3, computing unit 200-1 establishes a data transmission channel to the three-dimensional stacked storage device 300-3 via the data cache module 400, avoiding the redundant process of computing unit forwarding and shared cache synchronization in the traditional architecture, thereby reducing data transmission latency.

[0064] Figure 3 A schematic diagram of the architecture of a computing device provided in this application embodiment. Figure 2 .like Figure 3 As shown, the data caching module 400 provided in this application embodiment includes a cache slice group, which includes multiple cache slices (cache slice 400-1, cache slice 400-2, cache slice 400-3, and cache slice 400-4) that correspond one-to-one with multiple three-dimensional stacked storage devices. The first interface of each cache slice is connected to all computing units through a cross switch, and the second interface of each cache slice is connected to the corresponding three-dimensional stacked storage device.

[0065] In this embodiment, the data caching module 400 includes multiple cache slices, each of which corresponds one-to-one with a multiple three-dimensional stacked storage device (300-1, 300-2, 300-3, 300-4). Each cache slice acts as a "proxy" for the local main memory, allowing frequently accessed data to be pre-loaded into the cache, thus achieving data prefetching and reducing data transmission latency.

[0066] exist Figure 3 In the architecture shown, the first interface of each cache slice is connected to all computing units via a crossbar switch. By adjusting the connectivity of the crossbar switches, a data path can be established between any three-dimensional stacked storage device and any computing unit.

[0067] In the in-memory computing architecture provided in this application embodiment, the crossbar switch serves as a connection hub, possessing multi-port dynamic switching and high-concurrency connectivity capabilities. Based on data access requests initiated by computing units, the crossbar switch can construct a data transmission channel in real time from "computing unit - crossbar switch - target cache slice". This channel is a dedicated link, and channels between different access requests do not interfere with each other, thereby effectively avoiding latency accumulation caused by resource contention.

[0068] The dual-interface design of the cache slice in this embodiment of the application has a clear functional separation and coordination mechanism. The second interface serves as a dedicated interaction channel between the cache slice and the corresponding three-dimensional stacked storage device, mainly responsible for tasks such as data synchronization, prefetching, and updating. Through this interface, the cache slice can perceive changes in main memory data in real time, proactively prefetch frequently accessed data or data related to high-priority tasks to its local storage, and maintain consistency between the cache and main memory. This process is independent of the computing unit and does not require the resources of the first interface and the crossbar switch. The first interface is dedicated to responding to data access requests from the computing unit; after the crossbar switch completes the path configuration, the cache slice directly exchanges data with the requesting computing unit through this interface, thereby achieving high-speed delivery of cached data, or supporting temporary data pass-through when the cache misses.

[0069] In specific implementations, the first interface can be a high-speed serial computer expansion bus standard (PCIe) interface, a high-speed serial network interface (such as InfiniBand), or a customized dedicated high-speed interconnect protocol interface. These interfaces all have the characteristics of high bandwidth and low latency, which can support the frequent and large-scale data transmission needs between computing units and cache slices.

[0070] The second interface can be an interface that supports cache coherency protocols (such as CCIX), or a custom interface designed specifically for point-to-point high-speed data transmission. Such interfaces facilitate efficient data synchronization and consistency maintenance between cache slices and three-dimensional stacked storage devices.

[0071] Crossbar switches can employ a multi-port, high-bandwidth switching chip architecture, supporting dynamic configuration of port connections. This architecture can quickly switch connection paths between ports based on real-time data access requests, ensuring optimal data transmission links are always maintained between computing units and target cache slices. Furthermore, the switching chip can integrate error detection during data transmission and automatically correct errors using error correction codes (ECC), thereby guaranteeing the reliability and integrity of data transmission.

[0072] For example, each cache slice may also include an address resolution circuit for resolving the logical address in the data access request and determining the target three-dimensional stacked storage device corresponding to the logical address. Specifically, by dividing the logical address space into multiple contiguous and unique segments, each segment directly corresponding to a specific three-dimensional stacked storage device, the address resolution circuit can quickly locate the target storage device based on the segment to which the logical address in the data access request belongs, thereby reducing address resolution time. Furthermore, the address resolution circuit can also support dynamically updating the mapping relationship to adapt to changes in data distribution under different task scenarios, ensuring that the computing device maintains efficient data access capabilities in diverse application scenarios.

[0073] For example, each cache slice may also include a hit detection circuit to detect whether the target data pointed to by the data access request is already stored in static random access memory (SRAM). Specifically, if the target data already exists in SRAM, the hit detection circuit will immediately trigger a fast data return mechanism, thereby avoiding repeated readings of data from the three-dimensional stacked storage device and further optimizing access efficiency. Furthermore, cache slices can dynamically manage storage content through intelligent replacement algorithms, ensuring that frequently accessed data resides first and infrequently replaced data, maximizing cache utilization. Simultaneously, the data caching module supports flexible configuration, allowing adjustment of parameters such as cache slice size and address mapping rules according to application requirements, achieving a dynamic balance between performance, power consumption, and hardware resources.

[0074] For example, each cache slice may further include a data transfer circuit for copying target data from the target 3D stacked storage device to the static random access memory (SRAM) when the hit detection circuit detects that the target data is not stored in the SRAM. Specifically, after receiving a miss trigger signal from the hit detection circuit, the data transfer circuit sends a data read instruction to the target 3D stacked storage device through the second interface of the cache slice. The instruction carries the logical address corresponding to the target data. After receiving the target data returned by the target 3D stacked storage device, the data integrity is first verified. If the verification passes, the target data is written to the corresponding storage unit of the SRAM according to a preset storage rule through the internal data bus. At the same time, a data transfer completion signal is fed back to the hit detection circuit, and the cache status identifier is updated.

[0075] For example, the data caching module 400 may further include a request arbitration circuit, which is used to merge the requests of multiple computing units when it is detected that multiple computing units simultaneously send data access requests for the same data block in the target three-dimensional stacked storage device, and send the same data block to multiple computing units at once by broadcasting.

[0076] For example, the data caching module 400 may specifically include static random-access memory (SRAM). For instance, the data caching module 400 includes SRAMs that correspond one-to-one with the three-dimensional stacked storage devices, used to store frequently accessed data for the corresponding three-dimensional stacked storage devices. This architecture allows each computing unit to directly retrieve data from the corresponding cache slice when accessing data from non-local three-dimensional stacked storage devices, without having to read from the remote three-dimensional stacked storage device each time, thus significantly improving data access speed. Furthermore, the data caching module also supports dynamically adjusting the caching strategy, intelligently allocating cache space based on the access frequency and importance of the data, ensuring that critical data is cached first, further improving the overall performance of the system.

[0077] In one possible implementation, each of the aforementioned cache slices employs a data cache (DC cache) architecture based on SRAM media. Specifically, each DC cache is a complete, software-transparent hardware caching subsystem. It can automatically and transparently cache frequently accessed data blocks from the corresponding three-dimensional stacked storage device into a physical memory bank composed of a high-speed SRAM array.

[0078] Specifically, each cache slice can be called a DC Cache node. Each DC Cache node establishes a dedicated and unique data association mapping relationship with its corresponding 3D stacked storage device, becoming a dedicated data interaction agent for that 3D stacked storage device towards the computing unit group. This ensures efficient and targeted data transmission at the hardware level. Each DC Cache node supports configuring the cache block size according to the data read / write granularity of the 3D stacked storage device, which can be adapted to the storage granularity of the associated 3D stacked storage device, avoiding data splitting or splicing overhead caused by granularity mismatch.

[0079] In one possible implementation, the DC Cache node also has a built-in dedicated address translation logic unit, which is used to quickly parse the access address initiated by the computing unit and match it with the storage address of the local cached data, greatly reducing the address lookup time and improving the cache hit response speed.

[0080] In one possible implementation, the DC Cache node also has a built-in access trajectory statistics module, which is used to record in real time the access frequency, access time period and associated task type of the computing unit to the data of the associated three-dimensional stacked storage device. Combined with the task execution prediction information issued by the system, it actively fetches data blocks associated with high-frequency access and high-priority tasks from the associated three-dimensional stacked storage device and caches them. The number and granularity of the prefetched data blocks can be dynamically adjusted according to the task load to best meet the data access needs of the computing unit.

[0081] Regarding the data access process, when a computing unit initiates a non-local main memory data access request, the DC Cache node corresponding to the target 3D stacked storage device first performs a cache hit judgment. If the cache hits, the DC Cache node directly feeds back the target data to the requesting computing unit. Its data transmission latency is only the DC Cache's own read / write latency, which is much lower than the access latency of the remote 3D stacked storage device. If the cache misses, the DC Cache node will initiate a data read request to the associated 3D stacked storage device as a proxy. After obtaining the target data, it will synchronously cache the data block in the local DC Cache space and immediately feed it back to the requesting computing unit. Subsequent accesses to the data block by the same computing unit or other computing units can directly hit the cache, avoiding repeated remote main memory access requests.

[0082] In the aforementioned DC Cache implementation scheme, when multiple computing units concurrently access data in the same three-dimensional stacked storage device, they can all complete the data interaction through the DC Cache node corresponding to that three-dimensional stacked storage device, without directly connecting to the three-dimensional stacked storage device. This significantly reduces the concurrent access pressure on the three-dimensional stacked storage device and avoids the bandwidth congestion problem caused by multiple nodes accessing main memory simultaneously. At the same time, each DC Cache node has a built-in configurable cache replacement algorithm that supports multiple strategy selections such as Least Recently Used (LRU) and Least Frequently Used (LFU). It can intelligently adjust the retention rules of cached data based on data access frequency, data importance, and task priority. When the cache space reaches a threshold, it prioritizes the elimination of data blocks with low access frequency and non-core task association, further optimizing the overall system interaction efficiency and ensuring the cache hit rate of high-frequency critical data. From the perspective of caching strategy, it further improves the overall performance of system data access.

[0083] For example, computing unit 200-1 and computing unit 200-2 simultaneously request access to a task data block of a three-dimensional stacked storage device 300-3 associated with computing unit 200-3. The DC Cache corresponding to the three-dimensional stacked storage device 300-3 has prefetched and cached the data block based on historical access records. The access requests of the two computing units 200-1 and 200-2 are directly responded to by the DC Cache, without the need to initiate repeated read requests to the three-dimensional stacked storage device 300-3 associated with computing unit 200-3. Even if the two computing units 200-1 and 200-2 obtain the target data with the DC Cache read and write latency, the response latency caused by the three-dimensional stacked storage device 300-3 receiving multiple node requests at the same time is avoided, giving full play to the technical advantages of DC Cache in implementing cache slicing.

[0084] In this embodiment, the data caching module 400 also supports multiple data transmission modes, including point-to-point transmission, broadcast transmission, and multicast transmission. These transmission modes can be flexibly configured according to different application scenarios and needs to meet diverse data transmission requirements. For example, in scenarios requiring large-scale data synchronization, a broadcast transmission mode can be used to send data to all computing units at once, improving the efficiency of data synchronization.

[0085] For example, when multiple computing units request data at the same address, the requests can be merged and the results distributed via broadcast, thereby reducing the number of main memory accesses and further improving data transmission efficiency. This broadcast mechanism is particularly suitable for scenarios where multiple computing units process similar tasks in parallel or share the same dataset, effectively reducing the access pressure on three-dimensional stacked storage devices and improving the overall system throughput. For instance, during deep learning model training, multiple computing units may need to read the same batch of training data or model parameters simultaneously. In this case, the data can be sent to all requesting nodes at once via broadcast, avoiding the performance overhead caused by repeatedly accessing three-dimensional stacked storage devices.

[0086] In this embodiment, the three-dimensional stacked storage device adopts a globally unified addressing scheme. Each three-dimensional stacked storage device is assigned an independent address range (e.g., 0x0000~0xFFFF is assigned to three-dimensional stacked storage device 200-1, and 0x10000~0x1FFFF is assigned to three-dimensional stacked storage device 200-2). The software can directly locate the target data through address mapping without explicitly specifying the node ID, which can effectively simplify the programming complexity of cross-node data access.

[0087] In this embodiment, the computing unit group 200 and the three-dimensional stacked storage device group 300 are interconnected through the data caching module 400. This allows the computing units to directly access data from non-locally corresponding three-dimensional stacked storage devices without the need for data transfer and synchronization through intermediate computing units. This design simplifies the data access process, reduces data transmission latency and bandwidth consumption, and improves data access efficiency and overall system performance. Furthermore, the three-dimensional stacked storage devices use unified addressing, with each device allocated a different address space. This addressing method facilitates management and expansion, meeting the needs of large-scale in-memory computing architectures.

[0088] Figure 4 This is a flowchart illustrating a data access method provided in an embodiment of this application. Figure 4 The data access method shown is applied to the computing device 100 described above and includes the following steps:

[0089] Step S101: Receive a data access request sent by the computing unit. The data access request carries the logical address corresponding to the target data.

[0090] Step S102: Determine whether the target data has been stored in the data cache module based on the logical address;

[0091] Step S103: If the target data has been stored in the data cache module, then read the target data from the data cache module and return the target data to the computing unit;

[0092] Step S104: If the target data is not stored in the data cache module, the corresponding target 3D stacked storage device is determined according to the logical address, the target data is obtained from the target 3D stacked storage device through the data cache module, and returned to the computing unit.

[0093] Understandably, when a computing unit generates a data requirement, it will initiate a data access request. This data access request carries a logical address, which indicates the address of the target three-dimensional stacked storage device where the requested data resides.

[0094] For example, the data access request is a single load instruction. Specifically, in scenarios where the target data cache is hit, traditional solutions require additional copy and synchronization instructions to complete the data transmission to the requesting computing unit, which is cumbersome and has high latency. However, the embodiments of this application only require the execution of a single load instruction to achieve data transmission, and the latency period of this single load instruction is much smaller than the sum of the latency periods of the copy and synchronization instructions in the traditional solution. This can significantly reduce the time overhead of data transmission, greatly improve data access efficiency, effectively reduce the waiting latency of the computing unit, and further optimize the operating performance of the entire computing device.

[0095] For example, the process of determining the corresponding target three-dimensional stacked storage device based on the logical address includes: determining the logical address range to which the logical address belongs based on the unified continuous addressing of multiple three-dimensional stacked storage devices in the logical address space, and taking the three-dimensional stacked storage device corresponding to the logical address range as the target three-dimensional stacked storage device.

[0096] In this embodiment, the addresses of each three-dimensional stacked storage device are uniformly addressed, with each device having an independent address segment. This addressing method allows software to directly locate target data through address mapping without needing to specify the three-dimensional stacked storage device ID. When a computing unit initiates a data access request, it only needs to include the logical address in the request, and the device can determine the target three-dimensional stacked storage device where the data is located based on that address. This design simplifies the programming complexity of data access, enabling computing units to access data in different three-dimensional stacked storage devices in a unified manner, without needing to concern themselves with the actual storage location of the data. Simultaneously, the uniform addressing scheme also facilitates device expansion; when adding a new three-dimensional stacked storage device, only a new address segment needs to be allocated, without modifying the access logic of existing nodes. Upon receiving a data access request, the device parses the logical address, determines the corresponding three-dimensional stacked storage device, and further achieves efficient data transmission through the data caching module.

[0097] Optionally, the data access request also carries a computing unit identifier. This identifier is used to accurately distinguish the source of the request during data transmission, especially in scenarios where multiple computing units access the data concurrently, effectively avoiding request confusion or mis-sending of data. While parsing the logical address, the device can quickly locate the corresponding cache slice or 3D stacked storage device by combining the computing unit identifier, thereby ensuring the accuracy of the data transmission path. For example, when multiple computing units access data in the same 3D stacked storage device simultaneously, the computing unit identifier can help the data caching module distinguish the priority or scheduling order of different requests, avoiding data delays or loss caused by request conflicts.

[0098] It should be noted that, in specific implementations, the device can also identify the source of a data access request based on the physical interface through which it receives the request. For example, if each computing unit is connected to the data cache module via a dedicated interface, the device can directly trace the source of the request based on the interface number, without explicitly including the computing unit identifier in the data access request. This design helps simplify the encapsulation structure of data access requests and reduce protocol overhead, and is particularly suitable for closed hardware architectures where interfaces and computing units are fixedly bound.

[0099] Understandably, the device parses the logical address carried in the data access request and determines the corresponding target 3D stacked storage device based on the logical address. Subsequently, the device sends a data read command to the target 3D stacked storage device to obtain the required target data. This read command may contain address range information of the target data to ensure that the 3D stacked storage device can accurately identify and locate the data content. After the request is sent, the device waits for a response from the target 3D stacked storage device.

[0100] After the target data is obtained from the target 3D stacked storage device, the data is forwarded to the requesting computing unit through the data caching module 400, completing the entire data access process. The data caching module 400 enables cross-interconnection between the computing unit and the 3D stacked storage device, allowing the computing unit to directly access data from non-local corresponding 3D stacked storage devices without the need for data transfer and synchronization processes through intermediate computing units.

[0101] For example, the process of obtaining target data from a target 3D stacked storage device through a data caching module may include: sending a data read instruction to the target 3D stacked storage device through a cache slice connected to the target 3D stacked storage device; then, receiving the target data returned by the target 3D stacked storage device and storing the target data in the static random access memory of the cache slice; then, controlling a crossbar switch to connect the data transmission channel between the cache slice and the computing unit, and returning the target data to the computing unit through the data transmission channel.

[0102] Specifically, the data caching module 400 includes a cache slice group, which consists of multiple cache slices corresponding one-to-one with multiple three-dimensional stacked storage devices. The first interface of each cache slice is connected to all computing units via a crossbar switch, and the second interface of each cache slice is connected to the corresponding three-dimensional stacked storage device. By controlling the crossbar switch, the cache slice corresponding to the target three-dimensional stacked storage device and the computing unit are connected, thereby forming a data transmission channel, through which the target data is transmitted to the computing unit.

[0103] For example, before obtaining target data from the target three-dimensional stacked storage device corresponding to the logical address through the data caching module 400, it is determined whether the target data has been cached in the data caching module 400 according to the logical address; if the target data has been cached in the data caching module, the target data is read from the data caching module 400 and returned to the computing unit; if the target data has not been cached in the data caching module 400, the step of obtaining the target data from the target three-dimensional stacked storage device corresponding to the logical address through the data caching module is executed.

[0104] Specifically, after receiving a data access request and resolving the logical address, the device first checks whether the required target data is already cached in the data cache module 400. If the target data already exists in the data cache module 400, the device can directly read the data from the cache and establish a data transmission channel with the computing unit through a crossbar switch, quickly returning the data to the requesting node. This process does not require access to the 3D stacked storage device, thus significantly shortening the data access time, especially suitable for frequently accessed hot data scenarios. Conversely, if the target data is not cached in the data cache module 400, the device needs to perform the operation of retrieving data from the 3D stacked storage device. In this case, the device determines the corresponding target 3D stacked storage device based on the logical address and initiates a data read command to that node through the data cache module 400. Once the data arrives at the cache slice, the device stores it in the cache for subsequent access and simultaneously forwards the data to the computing unit through the data cache module 400, completing the entire data access process. This step aims to reduce the number of accesses to the 3D stacked storage device by utilizing cache to store hot data, thereby reducing data access latency and improving device performance.

[0105] For example, determining whether the target data has been stored in the data cache module is automatically performed by the hit detection circuit in the data cache module. The dedicated hit detection circuit can quickly determine the cache hit of the target data, shortening the response latency for data access while ensuring the accuracy of the determination result, providing a reliable basis for subsequent data transfer or direct reading operations.

[0106] To further improve cache utilization and overall device performance, the device can intelligently adjust its caching strategy based on data access frequency and importance. For example, for frequently accessed data or data associated with high-priority tasks, the device can prefetch it into the cache and allocate a larger cache space to ensure that this data can be accessed quickly. Simultaneously, the device can periodically clean up data that has not been accessed for a long time in the cache to free up cache space and provide storage resources for new data access requests. This dynamic cache management strategy can flexibly adjust cache content according to actual access conditions, thereby maximizing the effectiveness of the cache and improving the overall performance of the device.

[0107] For example, the process of determining whether target data has been stored in the data cache module 400 based on the logical address may include: determining the target three-dimensional stacked storage device corresponding to the logical address and the cache slice connected to the target three-dimensional stacked storage device based on the logical address; searching for the cache slice connected to the target three-dimensional stacked storage device; if the target data is found, confirming that the target data has been stored in the data cache module 400; if the target data is not found, confirming that the target data has not been stored in the data cache module 400.

[0108] Specifically, after resolving the logical address, the device first determines the target 3D stacked storage device corresponding to that address. Since each 3D stacked storage device is connected to a specific cache slice, the device can further locate the cache slice connected to the target 3D stacked storage device. Next, the device searches for the existence of the target data in the cache slice. This search process can be implemented by accessing the storage table or index structure of the cache slice to quickly determine whether the target data has been cached. If the target data is found, it is confirmed that the data cache module 400 has cached the data. At this time, the device can directly read the data from the cache slice and establish a data transmission channel with the computing unit through the crossbar switch to quickly return the data to the requesting node. Conversely, if the target data is not found, it is confirmed that the data cache module has not cached the data. At this time, the device needs to perform the operation of obtaining data from the 3D stacked storage device, that is, to send a data read command to the target 3D stacked storage device through the data cache module 400, and after the data arrives at the cache slice, store the data in the cache and forward it to the computing unit that requested the data.

[0109] For example, the process of obtaining target data from the target three-dimensional stacked storage device through the data caching module 400 and returning it to the computing unit may include: sending a data read instruction to the target three-dimensional stacked storage device through a cache slice in the data caching module 400 corresponding to the target three-dimensional stacked storage device, the data read instruction carrying a logical address; receiving the target data returned by the target three-dimensional stacked storage device and storing the target data in the static random access memory of the cache slice; controlling a crossbar switch to connect the data transmission channel between the cache slice and the computing unit, and returning the target data to the computing unit through the data transmission channel.

[0110] Specifically, when it is determined that target data needs to be retrieved from the 3D stacked storage device, the device uses the cache slice in the data cache module 400 corresponding to the target 3D stacked storage device to initiate a data read command. This cache slice, due to its direct connection with the target 3D stacked storage device, can efficiently convey the data access request. The data read command explicitly carries a logical address so that the 3D stacked storage device can accurately identify the required data content. After receiving the data read command, the 3D stacked storage device locates the corresponding data according to the logical address in the request and returns this data as a response to the cache slice that initiated the request. After receiving the target data returned by the target 3D stacked storage device, the cache slice first stores this data in its own storage space so that subsequent possible accesses can retrieve the data more quickly, reducing the access pressure on the 3D stacked storage device. Simultaneously, the cache slice will also forward the target data to the computing unit that initially initiated the data access request via a crossbar switch as needed, thereby completing the entire data acquisition and transmission process.

[0111] In one possible implementation, the method further includes: if multiple data access requests carrying the same logical address are received simultaneously from multiple computing units, the multiple data access requests are merged; and when the same data block corresponding to the logical address is obtained from the target three-dimensional stacked storage device or data cache module, the data block is broadcast to the multiple computing units at once.

[0112] Specifically, in practical applications, multiple computing units may simultaneously request the same data block from the same 3D stacked storage device. In this case, if the device receives multiple data access requests from computing units carrying the same logical address, it will merge these requests to avoid resource waste and performance degradation caused by repeated access to the 3D stacked storage device. Specifically, after receiving multiple requests, the device first compares and analyzes the logical addresses in the requests to confirm that they all point to the same data block in the same target 3D stacked storage device. Then, the device merges these requests into a single request task and initiates a data read command to the target 3D stacked storage device through the data caching module. After obtaining the target data, the device sends the data block to the corresponding computing units via a crossbar switch, based on the source information of each request before merging, ensuring that each requesting node can obtain the required data in a timely manner.

[0113] When the device needs to send acquired target data to multiple computing units simultaneously, it can use a broadcast method to achieve efficient data transmission. Specifically, after successfully acquiring the target data through the data caching module, the device controls the crossbar switch to establish data transmission channels with all relevant computing units. The establishment of these channels is based on the device's accurate identification and recording of the source information of each requesting node. Subsequently, the device broadcasts the target data through these established data transmission channels, ensuring that each computing unit receives the complete target data at the same time. By distributing data in one go through the broadcast method, the repetitive operation of transmitting data to each node individually is avoided, thereby significantly improving the efficiency of data transmission and the overall performance of the device. At the same time, the broadcast method also ensures the consistency of data acquired by each computing unit, providing convenience for subsequent parallel computing or data processing.

[0114] This device supports the merging and unified response to concurrent duplicate data access requests. Its workflow mainly includes two stages: request merging and data distribution. When the device detects multiple data access requests initiated by multiple computing units with the same logical address, it does not immediately initiate multiple read operations to the three-dimensional stacked storage device. Instead, it first merges these requests. After receiving and parsing these requests, the device confirms by comparison that they point to the same data range in the same three-dimensional stacked storage device. Subsequently, the system merges this group of requests into a unified internal access task and initiates a single data read request to the target three-dimensional stacked storage device through the data cache module 400. This mechanism effectively avoids repeated memory access operations for the same data, significantly saving memory bandwidth and controller resources. After successfully acquiring the target data through the data cache module 400, the device enters the data distribution stage. At this time, the system identifies all computing units requesting the data based on the previously recorded original request information (such as the identifiers of each computing unit or the corresponding input interfaces). Then, the device can control the crossbar switch to simultaneously establish data transmission channels to these nodes, and broadcast the target data to all relevant computing units at once.

[0115] For example, suppose computing units 200-1, 200-2, and 200-3 request main memory data at the same address almost simultaneously. Upon detecting this, the controller initiates only one read operation from the main memory. After retrieving the data, the controller, through a crossbar switch configuration, sends the same data packet in parallel to the receiving ports of the three nodes (200-1, 200-2, and 200-3), thus achieving efficient and consistent data distribution. This "merge-broadcast" mechanism not only reduces main memory access pressure and interaction latency but also significantly improves the overall system throughput and resource utilization efficiency by serving multiple requests with a single data transmission. It is particularly suitable for parallel computing scenarios where there is data sharing or synchronization requirements between computing units.

[0116] Figure 5 A schematic diagram of the architecture of a computing device provided in this application embodiment. Figure 3 . Figure 5The computing device architecture shown includes a computing unit group 200, a three-dimensional stacked storage device group 300, and a data cache module 400. The computing unit group 200 includes multiple computing units; the three-dimensional stacked storage device group 300 includes multiple 3D-DRAM nodes (3D-DRAM300-1, 3D-DRAM300-2, 3D-DRAM300-3, 3D-DRAM300-4) corresponding to each computing unit. The 3D-DRAM nodes in the three-dimensional stacked storage device group use a unified addressing system, and each 3D-DRAM node is allocated a different address space; the data cache module 400 includes a cache slice group, which includes multiple cache slices (Cache Slice 400-1, Cache Slice 400-2, Cache Slice 400-3, Cache Slice 400-4) corresponding to each 3D-DRAM node; the first interface of each cache slice connects to all computing units via a crossbar switch, and the second interface of each cache slice connects to the corresponding three-dimensional stacked storage device.

[0117] like Figure 5 In the illustrated computing architecture, each cache slice specifically includes an input buffer and SRAM. The input buffer receives various data access requests (covering target data read requests initiated by each computing unit) from the crossbar switch via the first interface of the cache slice, and simultaneously receives feedback data (i.e., target data) from the corresponding 3D stacked storage device via the second interface, temporarily storing the aforementioned requests and data. Given the randomness and burstiness of data access requests initiated by computing units, and the difference between the data transfer rate of 3D-DRAM and the port switching rate of the crossbar switch, the input buffer can avoid transmission interruptions caused by momentary data congestion, achieving a smooth data transition. SRAM (Static Random Access Memory), as the core high-speed storage component of the cache slice, leverages its fast read / write speed, low access latency, and lack of dynamic refresh requirement to perform the functions of cached storage and fast retrieval of target data. It is the core hardware for realizing the principle of cache locality and reducing data access latency.

[0118] Figure 6 This is a schematic diagram illustrating the address space partitioning of a three-dimensional stacked storage device group provided in an embodiment of this application. Figure 6As shown, each 3D-DRAM node in the three-dimensional stacked memory device group has the same spatial size. The 3D-DRAM nodes in the three-dimensional stacked memory device group adopt a unified addressing, and the address spaces of each 3D-DRAM node are independent and do not overlap. The specific address division is as follows: the address space of 3D-DRAM300-1 is 0x40300000~0x602FFFFF, the address space of 3D-DRAM300-2 is 0x60300000~0x802FFFFF, the address space of 3D-DRAM300-3 is 0x80300000~0xA02FFFFF, and the address space of 3D-DRAM300-4 is 0xA0300000~0xC02FFFFF.

[0119] The following will combine Figure 5 The computing device in the in-memory computing architecture shown and Figure 6 The unified address space layout of the three-dimensional stacked storage device group shown in the figure provides a detailed explanation and description of the data interaction and cache management methods provided in the embodiments of this application.

[0120] Example 1:

[0121] 1. Computing unit 200-1 in the computing unit group initiates a data read request. The request explicitly carries the address range information of the target data, namely 0xB0300000 to 0xB0300010.

[0122] 2. The address resolution unit in the computing device decodes and addresses the above logical address to determine that the corresponding physical storage node is 3D-DRAM300-4, that is, it identifies it as the target three-dimensional stacked storage device;

[0123] 3. Next, the cache control module will query the cache slice 400-4 associated with 3D-DRAM300-4 to determine whether the target data within the above address range has been cached there.

[0124] 4.1 If the corresponding target data is found in cache slice 400-4, it means that the cache has been hit. The control logic will directly read the data from cache slice 400-4 and return it to the computing unit 200-1 that initiated the request, thereby avoiding access to main memory.

[0125] 4.2 If the data is not cached in cache slice 400-4, the computing device will send a formal read instruction to 3D-DRAM 300-4 to obtain the required target data. After 3D-DRAM 300-4 returns the data, the device will send the data to computing unit 200-1 on the one hand, and write the data into cache slice 400-4 for caching on the other hand according to the caching strategy.

[0126] It should be further explained that during the process of caching the target data to cache slice 400-4, the computing device will perform corresponding operations according to the preset cache management strategy. For example, if there is still enough free storage space in cache slice 400-4, the target data will be directly written to the available area; if the remaining space in cache slice 400-4 is insufficient, the cache management unit will select some existing cached data with low access frequency or relatively low priority to replace it according to the cache replacement algorithm used (such as LRU, LFU, etc.), thereby freeing up the necessary storage space for the new data. After successfully storing the target data in cache slice 400-4, the computing device will update the internal storage table or metadata index structure of cache slice 400-4 accordingly to accurately record the address information and cache status of the target data so that subsequent accesses can retrieve it quickly. Subsequently, when the same computing unit or other computing units request access to data in the same address range again, the computing device can directly and quickly obtain the data from cache slice 400-4 without having to initiate a remote read request to 3D-DRAM 300-4 again, thereby significantly reducing the latency of data access and improving the overall storage-computing collaboration efficiency and system response performance.

[0127] Example 2:

[0128] 1. Assume that computing units 200-1, 200-2 and 200-3 initiate data read requests almost simultaneously, and all three requests point to the same target address range 0xB0300000~0xB0300010;

[0129] 2. Given that multiple computing units request access to the same address data, the request scheduling module in the computing device will merge these three requests into a unified data access request, and determine 3D-DRAM300-4 as the target three-dimensional stacked memory device to be accessed based on the logical address.

[0130] 3. The cache lookup unit determines whether the target data is already cached in the cache slice 400-4 corresponding to 3D-DRAM300-4;

[0131] 4.1 If the query finds that the required data already exists in cache slice 400-4, the computing device will directly return the target data to computing units 200-1, 200-2 and 200-3 respectively, so that multiple nodes can share the same cache copy;

[0132] 4.2 If the data is not yet cached in cache slice 400-4, the computing device sends a read command to 3D-DRAM 300-4. After obtaining the data, it distributes the returned target data to all requesting nodes (200-1, 200-2, 200-3) through a data broadcasting mechanism to reduce possible memory conflicts and duplicate transmissions. On the other hand, it writes the data into cache slice 400-4 for caching, so that it can be used by subsequent similar requests.

[0133] This application embodiment achieves efficient cross-interconnection between computing unit groups and three-dimensional stacked storage device groups by introducing a data caching module. Based on a globally unified 3D-DRAM addressing design, the data caching module can accurately parse the address information in data access requests, thereby locating the target 3D-DRAM node to be accessed. The complete data access process is as follows: When a computing unit initiates a data access request, the system first checks whether the requested data is already cached in the corresponding SRAM cache slice. If the cache hits, the SRAM directly returns the data to the requesting computing unit, eliminating the need for time-consuming barrier synchronization operations, greatly shortening the data access path and significantly improving system response speed and processing efficiency. If the cache misses, a data read request is automatically initiated to the target 3D-DRAM node. After obtaining the data, it is written into the SRAM for caching to facilitate subsequent access, and the data is also returned to the requesting node. In particular, when multiple computing units request data at the same address, SRAM can concurrently distribute the data to all requesting nodes through a data broadcasting mechanism after a hit. This mechanism effectively reduces repeated read operations of SRAM and significantly reduces the probability of memory conflicts, further optimizing the overall performance of the computing device.

[0134] SRAM (Static Random Access Memory) is the core high-speed storage component of cache slices. With its excellent characteristics of extremely high read and write speed, extremely low access latency, and no need for dynamic refresh, it undertakes the functions of temporary storage and fast response of target data in the cache system. It is a key hardware support for realizing the principle of cache locality, reducing the average data access latency, and improving the overall performance of the storage and computing system.

[0135] An exemplary embodiment of this application also provides a processor, the processor including the computing device described above.

[0136] Optionally, the processor can be a chip type with data processing capabilities, such as a graphics processing unit (GPU), a neural network processor (NPU), or a field-programmable gate array (FPGA). By integrating the aforementioned computing devices, these processors can significantly improve data processing efficiency, reduce data access latency, and optimize the overall performance of the device. For example, their application in GPUs can better support the data interaction needs in large-scale parallel computing scenarios; for NPUs, the introduction of computing devices helps improve the training and inference efficiency of deep learning models; and FPGAs, by integrating computing devices, can achieve more flexible data processing pipeline designs to meet the customized needs of specific application scenarios.

[0137] An exemplary embodiment of this application also provides a chip product, the chip product including a processor, the processor including the computing device described above.

[0138] Optionally, the chip product can be a GPU chip, an AI chip, or a DPU chip, etc. These chip products, with their included computing devices, are highly efficient in data processing and transmission. Taking GPU chips as an example, in scenarios requiring large amounts of data processing and transmission, such as graphics rendering, the data caching module in the computing device can quickly respond to data access requests from the computing unit. By using cache hit detection, it reduces the number of accesses to the three-dimensional stacked storage device, thus reducing data transmission latency. When the computing unit initiates a data access request, if the target data is in the data caching module, it can be directly retrieved from the cache, avoiding remote access to the three-dimensional stacked storage device and greatly improving data retrieval speed. For AI chips, during the training and inference of deep learning models, it is necessary to frequently read model parameters and training data. The computing device's globally unified addressing scheme allows the software to easily locate target data, while the data prefetching and caching strategy adjustment functions of the data caching module can intelligently allocate cache space based on the access frequency and importance of the data, prefetching high-frequency access data and high-priority task-related data into the local cache. When the computing unit needs this data, it can respond quickly, thereby improving the performance of the AI ​​chip when processing complex models. DPU chips are primarily used in data processing units, undertaking acceleration tasks for networking, storage, and security. The cross-connect design of the computing device simplifies the data access process, reduces data transmission latency and bandwidth consumption, enabling DPU chips to more efficiently complete data transmission and interaction tasks when processing large amounts of data, meeting their high-performance data access requirements.

[0139] An exemplary embodiment of this application also provides a computer device, the computer device including a processor, the processor including the computing means described above.

[0140] Optionally, the computer equipment can be a personal computer, workstation, game console, and some mobile devices (such as tablet computers, smartphones, etc.), or it can be an in-vehicle terminal device, smart home device, smart TV, smart robot, etc., or it can be a server, server cluster, artificial intelligence computing cluster, cloud computing cluster, etc., where artificial intelligence computing cluster can also be simply referred to as intelligent computing cluster or smart computing cluster, and this application does not limit it.

[0141] Please see Figure 7 This application also provides an electronic device, the electronic device 1100 including:

[0142] At least one processor 1101;

[0143] At least one memory 1102 is used to store at least one program;

[0144] At least one program is executed by at least one processor 1101 to implement the data access method of any of the foregoing embodiments.

[0145] This application also provides a computer program product, including a computer program or computer instructions, which are stored in a computer-readable storage medium. The processor of an electronic device reads the computer program or computer instructions from the computer-readable storage medium and executes the computer program or computer instructions, causing the electronic device to perform the data access method of any of the foregoing embodiments.

[0146] This application also provides a computer-readable storage medium storing computer-executable instructions for performing the data access method of any of the foregoing embodiments.

[0147] It should be understood that in the embodiments of this application, the processor can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0148] In addition, the processor may include one or more of the following: a central processing unit (CPU), a baseband processor, a digital signal processor (DSP), a microprocessor unit (MPU), a microcontroller unit (MCU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an artificial intelligence processor (AI processor), or a neural processing unit (NPU).

[0149] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the apparatuses and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0150] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0151] It should be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0152] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0154] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. If the above functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0155] Finally, it should be noted that the above embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in this application, or make equivalent substitutions for some of the technical features. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Finally, it should be noted that the above embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in this application, or make equivalent substitutions for some of the technical features. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application.

Claims

1. A computing device, characterized in that, include: Multiple computing units; Multiple three-dimensional stacked storage devices are respectively associated with multiple computing units. The multiple three-dimensional stacked storage devices adopt a unified continuous addressing in the logical address space. Each three-dimensional stacked storage device is allocated a unique and continuous address space. The data caching module is connected to each of the computing units and each of the three-dimensional stacked storage devices via cross switches. It is used to receive data access requests sent by any of the computing units, parse the logical address in the data access request, determine the target three-dimensional stacked storage device corresponding to the logical address according to the unified continuous addressing, copy the target data from the target three-dimensional stacked storage device and return it to the computing unit.

2. The computing device according to claim 1, characterized in that, The data caching module includes: Multiple cache slices are connected one-to-one with the multiple three-dimensional stacked storage devices; Each cache slice includes an independent static random access memory, a first interface, and a second interface. The first interface is connected to each computing unit via a crossbar switch, and the second interface is connected to the corresponding three-dimensional stacked storage device.

3. The computing device according to claim 2, characterized in that, Each of the cache slices also includes: The address resolution circuit is used to resolve the logical address in the data access request and determine the target three-dimensional stacked storage device corresponding to the logical address. A hit detection circuit is used to detect whether the target data pointed to by the data access request has been stored in the static random access memory; A data transfer circuit is used to copy the target data from the target three-dimensional stacked storage device to the static random access memory when the hit detection circuit detects that the target data is not stored in the static random access memory.

4. The apparatus according to claim 3, characterized in that, The data caching module also includes: An arbitration circuit is configured to merge the requests from multiple computing units when multiple computing units simultaneously send data access requests for the same data block in a target three-dimensional stacked storage device, and then broadcast the same data block to the multiple computing units at once.

5. The apparatus according to any one of claims 1 to 4, characterized in that, The three-dimensional stacked storage device is a three-dimensional stacked dynamic random access memory.

6. A data access method, applied to the computing device according to any one of claims 1 to 5, characterized in that, include: Receive a data access request sent by the computing unit, wherein the data access request carries the logical address corresponding to the target data; Based on the logical address, determine whether the target data has been stored in the data cache module; If the target data is already stored in the data cache module, then the target data is read from the data cache module and returned to the computing unit; If the target data is not stored in the data cache module, the corresponding target three-dimensional stacked storage device is determined according to the logical address, the target data is obtained from the target three-dimensional stacked storage device through the data cache module, and returned to the computing unit.

7. The method according to claim 6, characterized in that, Determining the corresponding target three-dimensional stacked storage device based on the logical address includes: Based on the unified continuous addressing of multiple three-dimensional stacked storage devices in the logical address space, the logical address range to which the logical address belongs is determined, and the three-dimensional stacked storage device corresponding to the logical address range is taken as the target three-dimensional stacked storage device.

8. The method according to claim 7, characterized in that, The data caching module retrieves the target data from the target 3D stacked storage device and returns it to the computing unit, including: Data read commands are sent to the target three-dimensional stacked storage device via a cache slice connected to the target three-dimensional stacked storage device. Receive the target data returned by the target three-dimensional stacked storage device, and store the target data in the static random access memory of the cache slice; The crossbar switch is controlled to connect the data transmission channel between the cache slice and the computing unit, and the target data is returned to the computing unit through the data transmission channel.

9. The method according to claim 6, characterized in that, Also includes: If multiple data access requests carrying the same logical address are received simultaneously from multiple computing units, then the multiple data access requests are merged. Once the same data block corresponding to the logical address is obtained from the target three-dimensional stacked storage device or the data cache module, the data block is broadcast to the multiple computing units at once.

10. The method according to claim 6, characterized in that, The determination of whether the target data has been stored in the data cache module is automatically performed by the hit detection circuit in the data cache module.

11. The method according to claim 6, characterized in that, The data access request is a single loading instruction.

Images