Storage device and data processing system

Abstract

The present disclosure provides a storage device and a data processing system, the device comprising: a logic interface module and a storage module, the logic interface module comprising a combination interface unit, an interface control unit, a D2D interface unit and a network on chip, the logic interface module connecting the storage module through the network on chip, wherein: the combination interface unit comprises a first interface controller and a second interface controller; the interface control unit is configured to configure the first interface controller and the second interface controller, so that the logic interface module connects a host device through a first interface and / or connects a remote device through a second interface; the D2D interface unit is configured to connect a computing power device through a D2D interface, so that the computing power device accesses the storage module through the logic interface module. According to the embodiments of the present disclosure, the interconnection of the storage device and various devices can be realized, and the scalability of the system is improved.

Description

Technical Field

[0001] This disclosure relates to the field of semiconductor technology, and in particular to a storage device and a data processing system. Background Technology

[0002] In the storage architecture design of modern computer systems, storage chips, such as high-bandwidth DRAM (Dynamic Random Access Memory), all follow the standardized interface protocols formulated by JEDEC (Joint Electron Device Engineering Council). Their core design goal is singular: to provide storage access channels for computing chips.

[0003] However, the core functions and protocols of the standard interfaces in memory chips are fixed and dedicated, used only for memory access, and cannot carry other types of communication; computing chips need to independently integrate various interconnect interfaces to connect memory chips and other devices, which leads to a reduction in the computing power density of computing chips and a complex overall system architecture. Summary of the Invention

[0004] This disclosure provides a storage device and a data processing system.

[0005] In a first aspect, this disclosure provides a storage device comprising a logical interface module and a storage module. The logical interface module includes a combined interface unit, an interface control unit, a D2D interface unit, and an on-chip network. The logical interface module is connected to the storage module via the on-chip network. The combined interface unit includes a first interface controller and a second interface controller. The interface control unit configures the first interface controller and the second interface controller, enabling the logical interface module to connect to a host device via a first interface and / or to a remote device via a second interface. The D2D interface unit connects to a computing device via a D2D interface, allowing the computing device to access the storage module through the logical interface module.

[0006] Secondly, this disclosure provides a data processing system comprising multiple computing cards, each computing card comprising a first storage device, a computing device, and a second storage device in a hybrid package. The first storage device is connected to the computing device via a D2D interface and to a host device via a first interface; the second storage device is connected to the computing device via a D2D interface and to the second storage devices of other computing cards among the multiple computing cards via a second interface; wherein the first storage device and the second storage device include the aforementioned storage devices.

[0007] Thirdly, this disclosure provides a data processing system comprising a hybrid-packaged third storage device, a computing device, a fourth storage device, and a fifth storage device. The third storage device is connected to the computing device via a D2D interface and to a host device via a first interface. The fourth storage device is connected to the computing device via a D2D interface and to the fifth storage device via a second interface. The third, fourth, and fifth storage devices include the aforementioned storage devices.

[0008] Fourthly, this disclosure provides an electronic device that includes the aforementioned storage device.

[0009] The embodiments provided in this disclosure can add a variety of interface units and interface control units to the storage device to realize the interconnection between the storage device and a variety of devices, thereby simplifying the interface structure of the computing device to improve the computing power density of the computing device, while simplifying the system architecture and improving the scalability of the system.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0012] Figure 1 This is a schematic diagram of the structure of a storage device provided in an embodiment of the present disclosure.

[0013] Figure 2 This is a schematic diagram of the encapsulation connection of a storage device provided in an embodiment of the present disclosure.

[0014] Figure 3 This is a schematic diagram of a chip stack for a storage device provided in an embodiment of the present disclosure.

[0015] Figure 4 This is a schematic diagram of a first operating mode of a storage device provided in an embodiment of the present disclosure.

[0016] Figure 5 This is a schematic diagram of a second operating mode of a storage device provided in an embodiment of the present disclosure.

[0017] Figure 6This is a schematic diagram of a third operating mode of a storage device provided in an embodiment of the present disclosure.

[0018] Figure 7 This is a schematic diagram of the structure of a data processing system provided in an embodiment of the present disclosure.

[0019] Figure 8 This is a schematic diagram of another data processing system provided in an embodiment of the present disclosure. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0022] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0024] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0025] As mentioned earlier, in memory chips, such as high-bandwidth DRAM, the core functions and protocols of standard interfaces are fixed and dedicated, used only for memory access. For example, in HBM (High Bandwidth Memory), a logic base die and multi-layer memory dies are stacked, connected via TSV (Through Silicon Via) and microbumps. The function of the logic base die is strictly limited to a "memory controller," connecting to computing chips such as GPUs (Graphics Processing Units) through the JEDEC-standardized HBM PHY (Physical Layer) interface. The protocol, timing, and function of the PHY interface are strictly defined, used only for transmitting memory read / write commands and data. Similarly, in GDDR (Graphics Double Data Rate) series chips, the connection to computing chips is also through a standardized GDDR PHY interface, and its interface function is similarly limited to memory access.

[0026] A common characteristic of storage architectures in related technologies is that their base chip or interface controller is a single-function "storage gateway," lacking system-level expansion and interconnect capabilities. DRAM also lacks internal computing functionality. This results in the following drawbacks:

[0027] Limited functionality, unable to adapt to modern heterogeneous computing architectures: The JEDEC standard interfaces of DRAM in related technologies (such as HBMPHY and GDDR PHY) have fixed and dedicated core functions and protocols, used only for memory access. They cannot carry other types of communication, making them a passive, single-function storage controller, and unable to integrate computing units to improve system energy efficiency.

[0028] The existence of a "memory wall" bottleneck results in significant data transfer overhead: In the storage architecture of related technologies, all data required by the computing unit must be completely transferred from DRAM to the local cache of the computing chip through a bandwidth-limited, high-latency interface for processing. This frequent data transfer process generates huge power consumption and access latency, becoming a "memory wall" that limits the overall energy efficiency and performance improvement of the system. This problem is particularly prominent for lightweight operations with high memory access and low computational density (such as data preprocessing, reduction, activation functions, etc.).

[0029] There are inherent "memory walls" and "expansion walls": Due to the single function of the interface, computing chips must independently integrate system-level interconnect interfaces such as PCIe (Peripheral Component Interconnect Express), SerDes (Serializer / Deserializer), and optical interconnects to connect to other chips, resulting in: reduced computing power density of computing chips: valuable computing chip area is occupied by various heterogeneous interfaces; limited scalability: the edge directions of computing chips are blocked by memory and system I / O interfaces, making it difficult to achieve multi-directional chiplet-level direct interconnection.

[0030] The system architecture is complex: computing, storage, and interconnect functions are distributed across different chips, with the computing chip serving as the central hub. The system bandwidth and latency bottlenecks are concentrated on the computing chip, which limits the expansion of system scale and the improvement of energy efficiency.

[0031] Inefficient data transfer path: In the storage architecture of related technologies, data transfer between host memory and device memory must pass through the computing chip, which occupies the D2D interface bandwidth of the computing chip and forms a system bottleneck.

[0032] Data exchange between storage units is complex: data migration across storage units requires reading from the computing chip first and then writing to the target storage unit, resulting in additional bandwidth consumption and latency.

[0033] Severe contention for system resources: Data transfer tasks and computing tasks share the interface bandwidth of the computing chip, leading to resource conflicts and performance degradation.

[0034] To address the system-level problems caused by the limited functionality of interfaces and the data transfer bottleneck of the "memory wall" in the storage architecture of related technologies, embodiments of this disclosure provide a storage device that can incorporate multiple interfaces and interface control units. This transforms the storage device from a single-function storage component into an intelligent storage and interconnection primitive with system-level scalability, thereby simplifying the interface structure of the computing device to increase its computing power density, while also simplifying the system architecture and improving the system's scalability.

[0035] The storage device according to embodiments of this disclosure can overcome the functional limitations of the JEDEC standard interface, enabling the storage device to support more functions beyond storage access; it can set up a near-memory computing unit to perform some computing tasks, significantly reducing data transfer volume, breaking through the "memory wall" limitation, and reducing system power consumption and access latency; it can offload system interconnect functions from the computing device to the storage device, thereby improving the computing power density and system scalability of the computing device; it can realize direct data transfer between the host device's memory and the local storage of the storage device, completely bypassing the computing device, reducing the bandwidth occupation of the computing device's interface by data transfer, reducing resource conflicts and improving the overall system efficiency; it can establish a direct data exchange channel between storage devices, supporting cross-node data migration and synchronization.

[0036] Figure 1 This is a schematic diagram of a storage device provided in an embodiment of the present disclosure. (Refer to...) Figure 1 The storage device includes a logic interface module 10 and a storage module 20. The logic interface module includes a combination interface unit 11, an interface control unit 12, a D2D interface unit 13, and an on-chip network 14. The logic interface module 10 is connected to the storage module through the on-chip network 14.

[0037] The combined interface unit 11 includes a first interface controller and a second interface controller.

[0038] The interface control unit 12 is used to configure the first interface controller and the second interface controller, so that the logic interface module connects to the host device through the first interface and / or connects to the remote device through the second interface.

[0039] The D2D interface unit is used to connect to the computing device via the D2D interface, so that the computing device can access the storage module through the logic interface module.

[0040] For example, the storage device may be a three-dimensional 3D chip structure, which may be called a memory chip or memory die (MemoryDie, or mem die). The logic interface module 10 may be a logic base chip (Base Die), or a logic base die. The storage module 20 may include a single layer of storage chips or multiple layers of storage chips stacked in the vertical direction. The storage chip (DRAMDie) may also be called a storage die, including multiple DRAM bank units 21 and corresponding multiple three-dimensional storage (3DDRAM) interfaces 22.

[0041] The number of stacked layers of the memory chip and the specific implementation technology of the 3D interface can be determined according to the memory type and process type, and adjusted according to bandwidth, cost and power consumption requirements. This disclosure does not impose any restrictions on these aspects. The specific implementation technologies of the 3D interface include, but are not limited to, microbump density, TSV process, and hybrid bonding process.

[0042] Figure 2 This is a schematic diagram of the encapsulation connection of a storage device provided in an embodiment of the present disclosure. Figure 3 This is a schematic diagram of a chip stack for a storage device provided in an embodiment of the present disclosure.

[0043] Among some possible implementations, refer to Figure 2 The logic interface module 10 and the storage module 20, that is, the logic substrate chip and the multi-layer storage chip, are directly connected through a high-speed, low-power three-dimensional storage interface 22; see reference. Figure 3 The logic interface module 10 and the storage module 20 are stacked vertically to achieve tight coupling between the storage module and the logic interface module.

[0044] In some possible implementations, the combined interface unit 11 in the logical interface module, also known as the multi-interface integrated combined physical layer (PCIe / SerDes Combo PHY), includes one or more independent first interface controllers and one or more independent second interface controllers.

[0045] In some possible implementations, the first interface controller can be a high-speed peripheral component interconnect (PCIe) controller, a protocol layer digital logic controller for the PCIe interface, used to implement PCIe bus communication, and the corresponding first interface is a PCIe interface; the second interface controller can be a serializer / deserializer (SerDes) controller, a physical layer transceiver controller for high-speed serial communication, used to implement serial-to-parallel conversion and data transmission and reception, and the corresponding second interface is a SerDes interface. Each second interface controller can be dynamically configured to operate in PCIe protocol mode or point-to-point direct connection mode.

[0046] In some possible implementations, the interface control unit 12 is used to configure the first interface controller and the second interface controller, so that the logic interface module connects to the host device through the first interface and / or connects to the remote device through the second interface, thereby improving the flexibility of the logic interface module in connecting to devices.

[0047] The interface control unit 12 can be a microcontroller unit (MCU) for flexible interface configuration and management. It can dynamically allocate the roles of the first and second interface controllers according to the system topology requirements, enabling various combinations of N PCIe interfaces + M SERDES direct connection ports (N, M ≥ 0). The interface control unit 12 also implements integrated control of the logic interface modules, managing their overall status, power consumption, interface configuration, and connection establishment.

[0048] In some possible implementations, through the configuration of the interface control unit 12, the logic interface module can connect to a host device via a first interface and / or to a remote device via a second interface. The host device can be the host itself or the central processing unit (CPU) within the host; the remote device can be any device connected to the transmission line corresponding to the second interface, such as another storage device or computing device.

[0049] It should be understood that those skilled in the art can set the first interface controller and the second interface controller, as well as the specific types of the corresponding first and second interfaces, according to the actual situation, and set the specific types of the host device and the remote device. For example, the second interface can be replaced by other high-speed serial interconnect interfaces instead of the SerDes interface. This disclosure does not limit this.

[0050] In some possible implementations, the D2D interface unit 13 in the logic interface module corresponds to a standardized D2D (Die-to-Die) interface. The D2D interface unit is used to connect to the computing device via the D2D interface, enabling the computing device to access the storage module through the logic interface module, achieving a high-speed, low-power direct connection. The computing device can be a computing chip, such as a graphics processing unit (GPU). This disclosure does not limit the specific device type of the computing device.

[0051] The D2D interface includes, but is not limited to, any one of the following: the Extra Short Reach SerDes physical layer XSR (Extra Short Reach SerDes) interface, the Universal Chiplet Interconnect Express (UCIe) interface, and the High Bandwidth Interconnect for Chiplet (HiPI) interface. It should be understood that those skilled in the art can configure the specific type of D2D interface according to actual circumstances, and this disclosure does not impose any limitations on this.

[0052] In some possible implementations, the logic interface chip further includes a three-dimensional bus interface, through which the on-chip network is connected to the three-dimensional storage interface of the logic substrate chip and the three-dimensional storage interface of the multilayer memory chip.

[0053] Reference Figure 1 and Figure 2 The Network on Chip (NOC) 14 in the logic interface module is connected to the three-dimensional storage interface of the logic substrate chip and the three-dimensional storage interface of the multi-layer storage chip via the three-dimensional bus interface 17, thereby directly connecting to the storage module. The NOC 14 acts as a "traffic hub" within the logic interface module, efficiently routing and scheduling all data flows and control signaling from the combined interface unit 11, the D2D interface unit 13, the three-dimensional storage interface 22, and other components in the logic interface module. This disclosure does not limit the specific structure of the NOC and the three-dimensional bus interface. In this way, the internal communication efficiency of the storage device can be further improved.

[0054] According to embodiments of this disclosure, multiple interface units and interface control units can be added to the storage device to achieve interconnection between the storage device and multiple devices, thereby simplifying the interface structure of the computing device to improve the computing power density of the computing device, while simplifying the system architecture and improving the scalability of the system.

[0055] The storage device according to embodiments of the present disclosure will now be described in detail.

[0056] In some possible implementations, the logical interface module includes a first working mode, a second working mode, and a third working mode. In the first working mode, the logical interface module is configured to connect to the host device through the first interface; in the second working mode, the logical interface module is configured to connect to the remote device through the second interface; in the third working mode, the logical interface module is configured to close the first and second interfaces and connect to the computing power device through a D2D interface.

[0057] For example, various interfaces of the logic interface module can be configured to enable the logic interface module to work in multiple modes, including a first working mode, a second working mode, and a third working mode.

[0058] Figure 4 This is a schematic diagram of a first operating mode of a storage device provided in an embodiment of the present disclosure.

[0059] Among some possible implementations, refer to Figure 4The first operating mode can also be called the PCIe storage mode. In the first operating mode, the logical interface module 10 is configured to connect to the host device 41 through one or more first interfaces, such as providing a PCIe x16 interface to connect to the host device, for the CPU of the host device 41 to access the storage module. In this case, the storage device can function as a standalone PCIe memory for the host device 41.

[0060] Figure 5 This is a schematic diagram of a second operating mode of a storage device provided in an embodiment of the present disclosure.

[0061] Among some possible implementations, refer to Figure 5 The second working mode can also be called the remote storage mode. In the second working mode, Figure 5 The storage device on the right is configured to connect to a remote device 52 via a second interface, wherein the remote device can be other computing cards, including at least one computing device and at least one storage device. Figure 5 In the middle, the remote device 52 includes a computing device and two storage devices.

[0062] In this case, all high-speed interfaces in the combined interface unit 11 are configured in SERDES direct connection mode, without providing PCIe interfaces to the host device, and are used for storage devices that other computing cards can access remotely via SERDES, thereby expanding storage capacity.

[0063] Figure 6 This is a schematic diagram of a third operating mode of a storage device provided in an embodiment of the present disclosure.

[0064] Among some possible implementations, refer to Figure 6 The third working mode can also be called the computing power storage mode. In the third working mode, the logic interface module 10 is configured to close the first and second interfaces and connect to the computing power device through the D2D interface. In this case, all high-speed interfaces are closed, and the computing power device accesses the storage device only through the D2D interface, using the storage device as a dedicated DRAM storage unit for the computing power device.

[0065] It should be understood that the operating modes of the storage device are not limited to the first, second, and third operating modes mentioned above. The first, second, and D2D interfaces in the logic interface module can be configured and combined in any way to enable interconnection and communication between the storage device and various devices.

[0066] This approach increases the flexibility of storage device configuration, simplifies system architecture, and enhances system scalability.

[0067] In some possible implementations, the logical interface module includes a fourth operating mode, in which the logical interface module is configured to connect to the host device through a first number of first interfaces and to connect to the remote device through a second number of second interfaces; wherein the first number and the second number are determined based on the interconnection topology information required by the data processing system to perform processing tasks, and the data processing system includes the storage device, the host device, and the remote device.

[0068] In other words, by utilizing the configurable interface types and quantities of the first interface (PCIe) and the second interface (SERDES) in the logical interface module, the interface control unit 12 can achieve flexible interface configuration and management, breaking the limitation of fixed interconnection topology of traditional computing cards. It allows for dynamic definition of system architecture, such as star, ring, and mesh topologies, based on application requirements (such as AI training and high-performance computing), to achieve more optimized communication performance.

[0069] In some possible implementations, the interface control unit 12 can realize multiple combination modes (N, M≥0) of N PCIe interfaces + M SERDES direct connection ports according to the interconnection topology information required by the data processing system to perform processing tasks. It is configured to connect to the host device through a first number of N first interfaces and to connect to the remote device through a second number of M second interfaces.

[0070] The data processing system includes storage devices, host devices, and remote devices; the first and second quantities can be determined based on the interconnection topology information. It should be understood that those skilled in the art can set the types and quantities of devices included in the data processing system, as well as the specific values ​​of the first and second quantities, according to actual circumstances; this disclosure does not impose any limitations in this regard.

[0071] This approach further enhances the flexibility of storage device configuration, improves system scalability, and enhances system communication performance.

[0072] Among some possible implementations, refer to Figure 1 The logical interface module 10 of the storage device further includes a Near-Memory Computing Unit (NCU) 15. This NCU 15 is used to: in active computing mode, read task data of a first computing task sent by the host device from the storage module and execute task processing; and / or, in collaborative computing mode, read task data of a second computing task sent by the computing power device from the storage module and execute task processing.

[0073] For example, a dedicated near-memory computing unit 15 can be integrated within the logic interface module 10. This near-memory computing unit 15 may be a general-purpose processor or not a complete general-purpose processor.

[0074] In some possible implementations, the near-memory computing unit 15 can be a hardware accelerator, such as a programmable vector processor, matrix multiplication accelerator, fixed-function tensor core, or reconfigurable logic, or a combination of multiple hardware accelerators to adapt to different application scenarios. The near-memory computing unit 15 can directly and with low latency access data in the 3D stacked memory chips and perform computations on-site through the high-bandwidth on-chip network 14 within the logic interface module 10.

[0075] In some possible implementations, the first computational task includes memory latency-sensitive computational tasks and / or data-movement-intensive computational tasks; the second computational task includes memory latency-sensitive computational tasks and / or data-movement-intensive computational tasks. Examples include vector addition / multiplication tasks, matrix addition / multiplication tasks, convolution tasks in neural network models, pooling tasks, etc. It should be understood that those skilled in the art can set the device type of the near-memory computing unit 15 and the type of computational task performed according to actual circumstances, and this disclosure does not impose any limitations in this regard.

[0076] In some possible implementations, the near-memory computing unit 15 can operate in active computing mode and / or collaborative computing mode. When the logical interface module of the storage device is connected to the host device via a first interface, the near-memory computing unit 15 can operate in active computing mode; when the logical interface module of the storage device is connected to the computing power device via a D2D interface, or to the computing power device in a remote device via a second interface, the near-memory computing unit 15 can operate in collaborative computing mode; the near-memory computing unit 15 can also operate in both active computing mode and collaborative computing mode simultaneously. This disclosure does not impose any limitations on this.

[0077] In some possible implementations, in active computing mode, the near-memory computing unit 15 can read the task data of the first computing task sent by the host device from the first interface, and perform task processing. The first computing task may include a computing task descriptor, including information such as the task operation type, the source data address of the task operands in the storage module, and the destination data address.

[0078] In some possible implementations, in collaborative computing mode, the near-memory computing unit 15 can read the task data of the second computing task from the storage module and perform task processing according to the second computing task sent by the computing device from the D2D interface.

[0079] In this way, data in DRAM can be processed with low latency and low power consumption, significantly reducing the amount of data to be moved and saving system bandwidth and power consumption.

[0080] In some possible implementations, under the collaborative computing mode, the near-memory computing unit is specifically used to: upon receiving a second computing task from the computing power device via the D2D interface, read the source data of the second computing task from the storage module according to the source data address; perform task processing on the source data according to the task operation type to obtain task result data; send the task result data to the destination data address in the storage module, and send a completion notification message of the second computing task to the computing power device via the D2D interface.

[0081] For example, in collaborative computing mode, the near-memory computing unit is used as a coprocessor for the computing power device to handle specific computing kernels offloaded by the computing power device, i.e., the second computing task. The second computing task may also include information such as the task operation type, the source data address and destination data address of the task operands in the storage module.

[0082] In some possible implementations, when collaborative computing between near-memory computing units is required, the computing device generates a second computing task and sends it to the near-memory computing unit (NCU) in the logic interface module via the D2D interface. For example, a task descriptor for "vector normalization" includes the source data address in the storage module and the destination data address where the result is stored.

[0083] In some possible implementations, after receiving the second computation task, the Near-Memory Computing Unit (NCU) reads the source data directly from the storage module via the on-chip Network Object Center (NOC) and the 3D storage interface, based on the source data address of the task operands in the storage module. It then performs task processing on the source data according to the task operation type (e.g., vector normalization) to obtain the task result data. After computation, the NCU writes the task result data directly back to the destination data address in the storage module via the NOC and the 3D storage interface, and sends a completion notification of the second computation task to the computing device via the D2D interface. This completes one near-memory computation process.

[0084] In this way, the source data and result data in the computing task do not need to be transmitted to the computing device through the D2D interface, which can greatly reduce the amount of data to be transported and significantly save system bandwidth and power consumption.

[0085] Among some possible implementations, refer to Figure 1The logical interface module also includes a Direct Memory Access (DMA) unit 16. This DMA unit 16 is used to: perform data transfer tasks on a target device, which includes at least one of the host device, the remote device, and the computing power device; and perform data exchange tasks between the storage module and other storage modules.

[0086] In other words, host devices, remote devices, and computing devices can all be used as target devices, sending data transfer tasks to the logical interface module of the storage device through corresponding interfaces; the logical interface module executes the data transfer tasks through the direct memory access unit 16, thereby achieving efficient direct data transfer.

[0087] In some possible implementations, other storage modules can also send data exchange tasks to the logical interface module of the storage device through corresponding interfaces. The logical interface module executes the data exchange task through the direct memory access unit 16, thereby achieving efficient direct data exchange.

[0088] This method can improve the efficiency of data transfer and data exchange.

[0089] The storage device according to the embodiments of this disclosure can be combined with a computing device in various ways to form a data processing system, serving as a computing node in a data processing cluster, or a supernode.

[0090] Figure 7 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this disclosure. (Refer to...) Figure 7 In some possible implementations, embodiments of the present disclosure also provide a data processing system, which includes a plurality of computing power cards 70, each computing power card 70 including a first storage device 71, a computing power device 73 and a second storage device 72 in a hybrid package.

[0091] The first storage device 71 is connected to the computing device via a D2D interface and to the host device via a first interface; the second storage device 72 is connected to the computing device via a D2D interface and to the second storage device of other computing cards among the plurality of computing cards via a second interface; wherein the first storage device and the second storage device include the aforementioned storage devices.

[0092] For example, two storage devices (hereinafter referred to as the first storage device and the second storage device) and one computing device (Chiplet) can be co-packaged as a single computing card. In each computing card, the first storage device 71 and the second storage device 72 are respectively connected to the computing device 73 via a D2D interface. The combined interface unit 11 in the logical interface module of the first storage device 71 is configured to connect to the host device via the first interface (i.e., the PCIe interface), for example, configured as a mode of 1 PCIe×16 (connected to the host device) + 3 SERDES direct connection; the combined interface unit 11 in the logical interface module of the second storage device 72 is configured to connect to the second storage device of other computing cards in multiple computing cards via the second interface (i.e., the SerDes interface), for example, configured as a mode of 4 SERDES direct connection.

[0093] It should be understood that Figure 7 The data processing system includes two computing cards 70 as an example, but those skilled in the art can set the number of computing cards in the data processing system according to the actual situation, such as 4, 8, etc., and this disclosure does not limit this.

[0094] In some possible implementations, the second storage devices 72 on multiple computing cards can be interconnected in a fully connected or ring topology via SERDES cables to form a high-bandwidth, low-latency inter-card data plane; the PCIe interfaces of the first storage devices 71 on multiple computing cards are connected to the same host device (host or switch) to form a control and management plane.

[0095] In this scenario, the data processing system appears as a unified supernode. The computing devices can directly and quickly access each other's memory through the inter-card data plane, process local data through the near-memory computing unit, and communicate with the host through the control plane.

[0096] This approach can improve the flexibility of data access and the efficiency of data processing within the data processing system.

[0097] Figure 8 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this disclosure. (Refer to...) Figure 8 According to embodiments of the present disclosure, a data processing system is also provided, which includes: a third storage device 81, a computing device 82 and a fourth storage device 83, and a fifth storage device 84, all in a hybrid package.

[0098] The third storage device is connected to the computing device via a D2D interface and to the host device via a first interface; the fourth storage device is connected to the computing device via a D2D interface and to the fifth storage device via a second interface; wherein the third, fourth, and fifth storage devices include the aforementioned storage devices.

[0099] For example, two storage devices (hereinafter referred to as the third storage device and the fourth storage device) and a computing device (Chiplet) can be co-packaged as a computing card 80. In the computing card 80, the third storage device 81 and the fourth storage device 83 are respectively connected to the computing device 82 through D2D interfaces. The combination interface unit 11 in the logical interface module of the third storage device 81 is configured to connect to the host device through the first interface (i.e., the PCIe interface); the combination interface unit 11 in the logical interface module of the fourth storage device 83 is configured to connect to the fifth storage device 84 through the second interface (i.e., the SerDes interface).

[0100] In this case, the fifth storage device can be connected to the computing card 80 via a SERDES cable, and the fifth storage device can be used as an extended DRAM storage unit of the computing device. Furthermore, the PCIe interface of the third storage device on the computing card is connected to the same host device to form a control and management plane.

[0101] In this way, the data storage capacity and data processing capabilities of the data processing system can be increased, and the flexibility of the data processing system for expansion can be improved.

[0102] Here are a few specific application examples.

[0103] Example 1: Multi-mode operation of a single DRAM memory cell (corresponding to the memory device in the embodiments of this disclosure).

[0104] Mode A (PCIe storage mode, corresponding to the first working mode): Configured to provide one PCIe x16 interface to the host for the host CPU to access DRAM storage units.

[0105] Mode B (Remote Storage Mode, corresponding to the second working mode): All high-speed interfaces are configured as SERDES direct connection mode, and no PCIe interface is provided to the host. This is used by other computing cards to remotely access DRAM storage units through SERDES to expand storage capacity.

[0106] Mode C (Computing chip storage mode, corresponding to the third working mode): All high-speed interfaces are turned off, and the computing chip accesses the DRAM storage unit only through the D2D interface.

[0107] Example 2: Unloading near-memory computing tasks.

[0108] Step 1: The computing chip sends a "vector normalized" task descriptor to the NCU of the logic base chip (Base Die) through the D2D interface. This descriptor contains the address of the source data in DRAM and the address where the result is stored.

[0109] Step 2: The NCU reads the source data directly through the NOC and 3D DRAM interfaces.

[0110] Step 3: The NCU performs normalization calculations locally.

[0111] Step 4: After the calculation is completed, the NCU writes the result data directly back to the specified location in the DRAM and sends a completion notification to the computing chip through the D2D interface.

[0112] Effect: Throughout the process, source data and result data do not need to be transmitted to the computing chip through the D2D interface, which greatly saves bandwidth and power consumption.

[0113] Example 3: Building a 4-card supernode system.

[0114] Step 1: Four computing cards, each containing one computing chip and two novel storage units as described in this disclosure (corresponding to the storage devices in the embodiments of this disclosure).

[0115] Step 2: Configure one storage unit A on each card as "1 PCIe x16 (connected to host) + 3 SERDES direct connection" mode. Configure the other storage unit B as "4 SERDES direct connection" mode.

[0116] Step 3: Connect the storage units B on the four cards in a fully connected or ring topology using SERDES cables to form a high-bandwidth, low-latency inter-card data plane.

[0117] Step 4: All cards are connected to the same host or switch through the PCIe interface of their storage unit A to form a control and management plane.

[0118] Effect: The system appears as a unified supernode to the outside world. The computing chips can directly and quickly access each other's memory through the inter-card data plane, process local data through the near-memory computing unit, and communicate with the host through the control plane.

[0119] According to the embodiments of the present disclosure, the storage device extends the logic substrate chip in the storage device into a multifunctional platform integrating storage control, near-memory computing, and system interconnection. Its advantages are as follows:

[0120] According to the embodiments of this disclosure, flexible heterogeneous interconnect topologies can be realized: by the configurable PCIe / SERDES interface type and quantity in the logical interface module, the limitation of fixed interconnect topologies of traditional computing power cards is broken, allowing the system architecture to be dynamically defined according to application requirements (such as AI training, high-performance computing), such as star, ring, and mesh topologies, to achieve optimal communication performance.

[0121] The embodiments of this disclosure can significantly reduce computational memory access latency and power consumption (a core advantage of near-memory computing). By setting up a near-memory computing unit (NOC), the amount of data transported can be eliminated: for tasks executed on the NCU, there is no need to transfer data to the computing chip via D2D or PCIe interfaces, completely eliminating the main power consumption and latency caused by this. It is estimated that for lightweight vector and matrix operations, up to 60%-80% of data transport power consumption can be saved.

[0122] Furthermore, the embodiments disclosed herein can drastically reduce latency: because the NCU and DRAM are connected through the chip's internal high-bandwidth, low-latency NOC and 3D interfaces, the access latency is reduced by an order of magnitude compared to accessing via an external interface. This is crucial for applications with high real-time requirements such as AI inference and data filtering.

[0123] Furthermore, the embodiments disclosed herein can improve the overall system throughput: the computing chip can focus on large-scale parallel computing, while offloading trivial and frequent memory access operations or specific computing kernels to the NCU, realizing the pipelined and parallelized computing tasks, and significantly improving the overall system throughput.

[0124] The embodiments of this disclosure can efficiently support the construction of supernodes: by using the logical interface module as a "system-on-a-chip" that integrates storage, near-memory computing and configurable interconnects, it is an ideal cornerstone for building large-scale, integrated supernode computing systems, greatly reducing the complexity and cost of building such systems.

[0125] The embodiments of this disclosure can overcome the functional limitations of traditional DRAM: elevating DRAM from a passive peripheral to an active system participant, providing a core foundational component for next-generation computing architectures, and creating conditions for computing chips to achieve higher computing power density by offloading interconnect functions.

[0126] The storage device according to an embodiment of the present disclosure includes:

[0127] 1. A paradigm shift from "passive storage" to "active computing and interconnect": Compared with traditional DRAM (such as HBM) which can only passively respond to read and write requests, the embodiments of this disclosure integrate configurable combined interface units and near-memory computing units in the logic interface module, enabling the memory to have system-level interconnect capabilities and preliminary data processing capabilities. This is a fundamental difference in architecture.

[0128] 2. Significant Energy Efficiency and Performance Improvement: By eliminating unnecessary data movement through near-memory computing, the system achieves energy efficiency benefits that traditional architectures cannot. Compared to traditional solutions (data is moved to computing chips for computation), the overall system energy efficiency (TOPS / W) and specific task latency of the embodiments of this disclosure can be improved by orders of magnitude when performing the same computing task.

[0129] 3. Unparalleled system construction flexibility: Compared with the fixed and rigid interconnection methods of traditional architectures, the configurable interconnection interface of the embodiments of this disclosure allows for dynamic optimization of system topology according to workload, realizing the flexibility of "one hardware, multiple architectures" and meeting different needs from edge computing to data center clusters.

[0130] 4. Reduce the burden on computing chips: By offloading complex system I / O functions from computing chips to this storage unit, computing chips can focus on computing to the extreme. Under the same semiconductor manufacturing photolithography mask size limit, the computing power density can be maximized, breaking through the key bottleneck of current AI computing power development.

[0131] It is understood that the various device embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further.

[0132] An electronic device is also provided according to embodiments of the present disclosure, the electronic device including the storage device described above.

[0133] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0134] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A storage device, characterized in that, The system includes a logical interface module and a storage module. The logical interface module comprises a combined interface unit, an interface control unit, a D2D interface unit, and an on-chip network. The logical interface module is connected to the storage module through the on-chip network, which is used for routing and scheduling data streams and control signaling within the logical interface module. The combined interface unit includes a first interface controller and a second interface controller; The interface control unit is used to configure the first interface controller and the second interface controller, so that the logic interface module connects to the host device through the first interface and / or connects to the remote device through the second interface; The D2D interface unit is used to connect to the computing device via the D2D interface, so that the computing device can access the storage module through the logic interface module.

2. The apparatus according to claim 1, characterized in that, The logic interface module includes a first working mode, a second working mode, and a third working mode. In the first operating mode, the logic interface module is configured to connect to the host device through the first interface; In the second operating mode, the logic interface module is configured to connect to the remote device through the second interface; In the third working mode, the logic interface module is configured to close the first interface and the second interface, and connect to the computing device through the D2D interface.

3. The apparatus according to claim 1, characterized in that, The logical interface module includes a fourth working mode. In the fourth working mode, the logical interface module is configured to connect to the host device through a first number of first interfaces and to connect to the remote device through a second number of second interfaces. The first quantity and the second quantity are determined based on the interconnection topology information required by the data processing system to perform processing tasks. The data processing system includes the storage device, the host device, and the remote device.

4. The apparatus according to claim 1, characterized in that, The logical interface module also includes a near-memory computing unit, used for: In active computing mode, according to the first computing task sent by the host device, the task data of the first computing task is read from the storage module and the task processing is performed. And / or, In collaborative computing mode, according to the second computing task sent by the computing power device, the task data of the second computing task is read from the storage module and the task processing is executed.

5. The apparatus according to claim 4, characterized in that, The second computation task includes a task operation type, the source data address and destination data address of the task operands in the storage module. In the collaborative computation mode, the near-memory computation unit is specifically used for: When a second computing task is received from the computing power device through the D2D interface, the source data of the second computing task is read from the storage module according to the source data address; The source data is processed according to the task operation type to obtain task result data; The task result data is sent to the destination data address in the storage module, and the completion notification information of the second computing task is sent to the computing device through the D2D interface.

6. The apparatus according to claim 4, characterized in that, The near-memory computing unit includes a hardware accelerator; the first computing task includes memory access latency-sensitive computing tasks and / or data movement intensive computing tasks; the second computing task includes memory access latency-sensitive computing tasks and / or data movement intensive computing tasks.

7. The apparatus according to claim 1, characterized in that, The logical interface module also includes a direct memory access unit, used for: Perform a data transfer task on a target device, wherein the target device includes at least one of the host device, the remote device, and the computing power device; Perform data exchange tasks between the storage module and other storage modules.

8. The apparatus according to claim 1, characterized in that, The logic interface module is a logic substrate chip, the storage module includes multiple layers of storage chips stacked vertically, and the computing power device includes computing power chips. The multilayer memory chip and the logic substrate chip are connected through a three-dimensional memory interface and are stacked in the vertical direction.

9. The apparatus according to claim 8, characterized in that, The logic interface chip also includes a three-dimensional bus interface, through which the on-chip network is connected to the three-dimensional storage interface of the logic substrate chip and the three-dimensional storage interface of the multi-layer storage chip.

10. The apparatus according to claim 1, characterized in that, The first interface controller includes a high-speed peripheral component interconnect PCIe controller, and the first interface includes a PCIe interface; or, the second interface controller includes a serializer-deserializer SerDes controller, and the second interface includes a SerDes interface.

11. A data processing system, characterized in that, It includes multiple computing cards, each of which includes a first storage device, a computing device, and a second storage device in a hybrid package. The first storage device is connected to the computing device via a D2D interface and to the host device via a first interface; The second storage device is connected to the computing power device via a D2D interface, and is also connected to the second storage devices of other computing power cards among the multiple computing power cards via a second interface; Wherein, the first storage device and the second storage device include the storage device according to any one of claims 1-10.

12. A data processing system, characterized in that, This includes a hybrid-packaged third storage device, a computing device, a fourth storage device, and a fifth storage device. The third storage device is connected to the computing device via a D2D interface and to the host device via a first interface; The fourth storage device is connected to the computing device via a D2D interface and to the fifth storage device via a second interface; The third storage device, the fourth storage device, and the fifth storage device include the storage device according to any one of claims 1-10.

Images