High bandwidth memory controller and data transfer mapping method, computing device, and system

Abstract

The present disclosure relates to a high bandwidth memory controller and data transmission mapping method, computing device and system. The controller comprises at least one data transmission port configured to connect a high bandwidth memory to establish a physical channel for data interaction therebetween, the physical channel comprising a first pseudo channel and a second pseudo channel; a write data interface configured to receive write data transmitted by an on-chip bus, a data bit width of the write data interface being equal to a data bit width of the physical channel, the write data interface being divided into two bit-continuous interface units from low bits to high bits, respectively a first and a second write data interface unit; and a pseudo channel mapping module configured to map low bit write data output by the first write data interface unit to the first pseudo channel of the physical channel, and to map high bit write data output by the second write data interface unit to the second pseudo channel of the physical channel. The architecture can improve the transmission performance of the controller for narrow write data.

Description

Technical Field

[0001] This disclosure relates to the field of memory control technology, and more specifically, to a high-bandwidth memory controller, a data transfer mapping method for a high-bandwidth memory controller, a computing device, and a computing system. Background Technology

[0002] With the increasing demands for computing power from artificial intelligence and high-performance computing scenarios, High-Bandwidth Memory (HBM), with its stacked DRAM architecture and ultra-high bandwidth parallel data transfer capabilities, has become an indispensable storage component for computing devices such as GPUs and NPUs. The High-Bandwidth Memory Controller (HBM Controller), as a key bridging module connecting computing units or cache units in computing devices with the HBM, is used to complete memory access protocol conversion, data routing and transmission scheduling between upstream and downstream components, and to achieve data interaction with the HBM through its physical channel.

[0003] In existing HBM controller write operations, a fixed-granularity interleaving mapping strategy is typically used. This involves dividing the write data transmitted via the on-chip bus into segments of a set granularity and then alternately mapping them to the two pseudo-channels (PCs) of the same physical channel. However, due to HBM protocol specifications, HBM does not support data masking. Therefore, for narrow write data smaller than the physical channel data width (e.g., 512 bits), the HBM controller cannot directly update local bit segments and must rely on a "Read-Modify-Write (RMW)" mechanism. This mechanism first reads the complete row data corresponding to the target address from HBM. The row data has the same data width as the physical channel and completely covers the two pseudo-channels of the physical channel. Then, within the HBM controller, the narrow write data overwrites the corresponding bit segments of the row data. Finally, the updated complete row data is written back to HBM. In this conventional alternating mapping method, because the fragments of narrow write data are forced to be distributed to two pseudo channels on the same physical channel, both pseudo channels have to perform the complete RMW process, which limits the write throughput performance of HBM. Summary of the Invention

[0004] One object of this disclosure is to provide a new data transmission mapping scheme for HBM controllers to improve the transmission performance of HBM controllers for narrow write data.

[0005] According to a first aspect of this disclosure, a high-bandwidth memory controller is provided, comprising: At least one data transmission port, each of the data transmission ports being configured to connect to high-bandwidth memory to establish a physical channel for data interaction between the high-bandwidth memory controller and the high-bandwidth memory; wherein the physical channel includes a first pseudo-channel and a second pseudo-channel; A write data interface configured to receive write data transmitted via the on-chip bus; wherein the data bit width of the write data interface is equal to the data bit width of the physical channel, and the write data interface is divided along the bit position from the least significant bit to the most significant bit into two consecutive bit interface units, namely a first write data interface unit and a second write data interface unit; and, A pseudo-channel mapping module is configured to map the low-order write data output by the first write data interface unit to a first pseudo-channel of the physical channel, and to map the high-order write data output by the second write data interface unit to a second pseudo-channel of the physical channel.

[0006] Optionally, the high-bandwidth memory controller further includes a first pseudo-channel buffer, a second pseudo-channel buffer, and a scheduling module; The pseudo-channel mapping module is configured to: write the low-order write data output by the first write data interface unit into the first pseudo-channel buffer, and write the high-order write data output by the second write data interface unit into the second pseudo-channel buffer. The scheduling module is configured to use the amount of data transmitted in a single pseudo-channel as the scheduling unit, to send the low-order write data in the first pseudo-channel buffer to the first pseudo-channel port corresponding to the data transmission port, and to send the high-order write data in the second pseudo-channel buffer to the second pseudo-channel port corresponding to the data transmission port.

[0007] Optionally, the data bit width of both the physical channel and the write data interface is 512 bits; the lower 256 bits of the write data interface constitute the first write data interface unit, and the higher 256 bits constitute the second write data interface unit.

[0008] According to a second aspect of this disclosure, a data transfer mapping method for a high-bandwidth memory controller is provided, the high-bandwidth memory controller being used to connect to high-bandwidth memory to establish a physical channel for data interaction between the high-bandwidth memory controller and the high-bandwidth memory, the physical channel including a first pseudo-channel and a second pseudo-channel; the method includes: Receive write data transmitted through the write data channel of the on-chip bus; wherein the data bit width of the write data channel is equal to the data bit width of the physical channel, and the write data channel is divided into two consecutive bit segments along the bit from low bit to high bit, namely the first bit segment corresponding to the first pseudo channel and the second bit segment corresponding to the second pseudo channel. Determine the bit field to which the written data belongs: If the write data belongs to a single bit segment, the write data is mapped to a single pseudo channel corresponding to the single bit segment for data transmission; wherein, the single bit segment is the first bit segment or the second bit segment.

[0009] Optionally, if the write data belongs to a single bit segment, the step of mapping the write data to a single pseudo-channel corresponding to the single bit segment for data transmission includes: When the number of bits in the write data is equal to the data width of the single bit segment, the write data itself is mapped to the single pseudo channel corresponding to the single bit segment for data transmission.

[0010] Optionally, if the write data belongs to a single bit segment, the step of mapping the write data to a single pseudo-channel corresponding to the single bit segment for data transmission includes: When the number of bits in the written data is less than the data width of the single bit segment, the complete data block to which the written data belongs and matches the single pseudo channel is read from the high-bandwidth memory; wherein, the number of bits in the data block is equal to the data width of the single bit segment; Based on the offset position of the written data within the data block, the corresponding data fragment in the data block is replaced by the written data. The data block that has been replaced is mapped to a single pseudo-channel corresponding to the single bit segment for data transmission, so as to write the complete data block that has been replaced back to the high-bandwidth memory.

[0011] Optionally, after determining the bit field corresponding to the written data, the method further includes: If the written data includes a first data portion belonging to the first bit segment and a second data portion belonging to the second bit segment, then the first data portion is mapped to the first pseudo channel for data transmission, and the second data portion is mapped to the second pseudo channel for data transmission.

[0012] Optionally, if the written data belongs to a single bit field, the method further includes: Schedule another pseudo-channel besides the single pseudo-channel to transmit data from other memory access operations that are independent of the write operation corresponding to the write data.

[0013] According to a third aspect of this disclosure, another high-bandwidth memory controller is also provided, which is configured to perform the method described according to a first aspect of this disclosure.

[0014] According to a fourth aspect of this disclosure, a computing device is provided, the computing device comprising: Multiple computing units; and, A high-bandwidth memory controller, wherein the high-bandwidth memory controller is as described in the first or third aspect of this disclosure, and the high-bandwidth memory controller interacts with the computing unit via an on-chip bus.

[0015] According to a fifth aspect of this disclosure, a computing system is provided, the computing system comprising: high-bandwidth memory and a computing device according to a fourth aspect of this disclosure, the high-bandwidth memory being connected to a high-bandwidth memory controller of the computing device to establish a physical channel for data interaction between the high-bandwidth memory and the high-bandwidth memory controller of the computing device.

[0016] This embodiment of the disclosure divides the write data interface into two consecutive bit segments: a first write data interface unit and a second write data interface unit. The consecutive low-bit write data output from the first write data interface unit is mapped to a first pseudo-channel of the physical channel, and the consecutive high-bit write data output from the second write data interface unit is mapped to a second pseudo-channel of the physical channel. Through this segmented mapping method, in narrow write scenarios, the narrow write data will be completely mapped to a single pseudo-channel. Furthermore, if the number of bits in the narrow write data is equal to the data width of the single pseudo-channel, the write operation can be performed directly without triggering the RMW process. Therefore, the RMW overhead when HBM processes narrow write data can be effectively reduced, thereby improving the transmission efficiency and overall performance of the HBM controller for narrow write data to a certain extent.

[0017] The features and advantages of the embodiments of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of these embodiments.

[0019] Figure 1 This is a schematic diagram of the structure of a computing system to which the methods of the embodiments of this disclosure can be applied; Figure 2 This is a schematic diagram of data transmission mapping for the HMB controller in related technologies; Figure 3 This is a schematic diagram of the microarchitecture of an HBM controller according to some embodiments; Figure 4 This is a schematic diagram of data transmission mapping of an HMB controller according to some embodiments; Figure 5 This is a microarchitecture diagram of an HBM controller according to some other embodiments; Figure 6This is a flowchart illustrating a data transmission mapping method for an HBM controller according to some embodiments; Figure 7 This is a schematic diagram of the structure of a computing device according to some embodiments. Detailed Implementation

[0020] Various exemplary embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0021] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the embodiments of this specification or their application or use.

[0022] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0023] This disclosure relates to a data transfer mapping scheme for HBM controllers. Figure 1 This is a schematic diagram of the structure of a computing system to which the data transmission mapping scheme of the HBM controller provided in the embodiments of this disclosure can be applied. Figure 1 As shown, the computing system 1000 may include a computing device 1100, a control device 1200, and a high-bandwidth memory (HBM) 1300. The computing device, control device, and HBM can be connected via a bus network or other means. The HBM controller can be integrated inside the computing device 1100 and is responsible for the data interaction between the computing units in the computing device 1100 and the HBM.

[0024] High-bandwidth memory (HBM) is a stacked DRAM high-speed storage device that can store computational data, weight data, program instructions, etc. required for computing tasks, and can be read or written by control devices and computing devices. HBM adopts a 3D stacked DRAM architecture and has ultra-high parallel data transmission bandwidth, making it suitable for use as a large-capacity external high-speed memory for computing devices and / or control devices.

[0025] This control device is used to issue computing tasks and schedule and control computing devices to execute corresponding processes, thereby realizing system functions such as artificial intelligence inference, scientific computing, and graphics processing. For example, the control device can be a Central Processing Unit (CPU); the computing device can be a Graphics Processing Unit (GPU), a General-Purpose Graphics Processing Unit (GPGPU), a Neural-network Processing Unit (NPU), or a Tensor Processing Unit (TPU), etc. This computing device can also be referred to as an AI chip, computing chip, or processor.

[0026] Continue as Figure 1 As shown, the computing device 1100 may include a computing unit 1110 and a cache 1120. The computing unit may be the core component in the computing device that performs computing tasks, and may also be referred to as a computing core, arithmetic core, or processing core.

[0027] The data access latency of the internal cache of this computing device is lower than that of the external HBM, but its storage capacity is much smaller than that of the HBM. The cache is used to temporarily store recently accessed instructions and data, thereby reducing the overhead of external memory access and improving the overall processing efficiency of the computing device. For example, the computing device can be configured with a multi-level cache architecture (at least two levels), such as including a level 1 cache (L1 Cache) and a level 2 cache (L2 Cache), or including a level 1 cache (L1 Cache), a level 2 cache (L2 Cache), and a level 3 cache (L3 Cache), etc. The level 2 cache or level 3 cache, as the last level cache (LLC), is the cache with the largest chip content and the closest to the external HBM. It can interact with the HBM controller through the on-chip bus to read and write instructions and / or data to the HBM. For example, each computing unit can be configured with an independent level 1 cache, which is exclusively used by a single computing unit; alternatively, multiple computing units within the computing device can be divided into multiple computing unit groups (e.g., SPCs), with each computing unit group sharing the level 1 cache. The last-level cache can be shared by all or part of the computing units of a computing device; it can also be called a shared cache or the last-level cache. Each computing unit can directly initiate a memory access request, or forward the memory access request through each level of cache, and finally complete the data reading and writing to the external HBM through the HBM controller.

[0028] The HBM controller 1130 is equipped with a bus interface for connecting to the on-chip bus inside the computing device. The on-chip bus within the computing device contains five independent transmission channels: a write address channel for transmitting write requests, a write data channel for transmitting write data, a write response channel for reporting write transaction completion status, a read address channel for transmitting read requests, and a read data channel for reporting read data. Correspondingly, the HBM bus interface includes various interfaces that interface with each of the on-chip bus channels, such as a write data interface that interfaces with the write data channel, and the write data interface has the same data bit width as the write data channel. The HBM controller 1130 also includes at least one data transmission port, each consisting of multiple data pins. Each data transmission port corresponds to a physical channel, and the data bit width of this physical channel is equal to the data bit width of the write data interface, for example, 512 bits. It should be noted that the number of data pins on the data transmission port is not equal to the data bit width of the physical channel; the HBM controller can output 512 bits of data in multiple clock cycles. For example, a single data transmission port is configured with 64 parallel data pins, using a double-edge triggering mechanism, and transmits data in parallel in 8 steps within 4 clock cycles, splicing them together to form a 512-bit wide physical channel.

[0029] When performing write operations, the HBM controller 1130 employs a dual pseudo-channel interleaving mapping strategy. Taking an example where the physical channel and write data interface width are both 512 bits, and the basic fragmentation granularity is 32 bits, as follows... Figure 2 As shown, the HBM controller segments the write data output from the write data interface into 32-bit granularity segments from least significant bit to most significant bit. These segments are then alternately distributed to the first pseudo-channel PC0 and the second pseudo-channel PC1 of the physical channel. In this mapping method, if the number of bits in the write data exceeds 32 bits, the data segments will inevitably be distributed across the two pseudo-channels. This means that 64-bit, 128-bit, or 256-bit narrow write data will simultaneously occupy both pseudo-channels. Combined with the limitation that the HBM protocol does not support the Data Mask signal, both pseudo-channels of the physical channel must execute the RMW process during narrow write operations. That is, the resources of both pseudo-channels are exclusively occupied by a single narrow write operation, and cannot handle other independent memory access requests in parallel. Therefore, for narrow write operations, the dual pseudo-channel interleaving mapping strategy leads to RMW transaction bandwidth loss, significantly reducing HBM's write throughput performance.

[0030] Therefore, this disclosure proposes an improved data transmission mapping scheme for HBM controllers to improve the transmission efficiency and overall performance of HBM controllers for narrow write data to a certain extent.

[0031] The following combination Figure 1 computing systems and Figure 2 The HBM controller in the present disclosure illustrates various embodiments of this disclosure.

[0032] <First Embodiment> Figure 3 A schematic diagram of the microarchitecture of an HBM controller according to some embodiments is shown. Figure 3 As shown, in this embodiment, the HBM controller 1130 includes a write data interface 1131, at least one data transmission port 1132, and a pseudo-channel mapping module 1133.

[0033] In this embodiment, data transmission port 1132 is used to connect to external high-bandwidth memory 1300. The HBM controller 1130 establishes a physical channel with the HBM through data transmission port 1132 to provide the underlying transmission carrier for write data fragment mapping. Each data transmission port 1132 corresponds to one physical channel. For example, the HBM controller 1130 has 16 data transmission ports to establish 16 physical channels with the HBM. Figure 1 Only one data transmission port is shown schematically. The multiple physical channels between the HBM controller 1130 and the HBM are independent of each other, do not interfere with each other, and can transmit data in parallel.

[0034] Each physical channel is logically divided into a first pseudo-channel PC0 and a second pseudo-channel PC1. The two pseudo-channels within the physical channel have their own independent read / write transaction scheduling, RMW processing, and data caching resources, and can work in parallel and independently.

[0035] The first pseudo-channel PC0 and the second pseudo-channel PC1 of a single physical channel can also be configured with bidirectional transmit and receive logic, which can simultaneously carry read and write transaction data transmission through the two pseudo-channels, and distinguish between read and write data streams through internal timing arbitration.

[0036] The HBM controller 1130 selects the corresponding physical channel through address decoding based on the target address in the write request for the transmission of the write data corresponding to the write request. The write request is transmitted through the write address bus, and the write data is transmitted through the write data bus.

[0037] Each data transmission port consists of multiple data pins. As mentioned above, the logic data width of the physical channel does not represent the number of data pins of the data transmission port. The data transmission port can achieve the complete logic data width of the physical channel by splicing multiple clock cycles.

[0038] The write data interface 1131 is configured to receive write data output from the write data channel of the on-chip bus. The data bit width of the write data interface 1131 is the same as the data bit width of a single physical channel, for example, 512 bits.

[0039] In this embodiment, the write data interface 1131 is segmented along the bit from low to high to form two consecutive non-overlapping bit intervals, which are defined as the first write data interface unit 1131_1 and the second write data interface unit 1131_2, respectively. The first write data interface unit 1131_1 corresponds to the transmission interval of the first pseudo-channel PC0 of the physical channel, and the second write data interface unit 1131_2 corresponds to the transmission interval of the second pseudo-channel PC1 of the physical channel. In some examples, the two pseudo-channels of the physical channel are divided with equal bit width, each occupying half of the data bit width of the physical channel; correspondingly, the two write data interface units of the write data interface are also divided with equal bit width, each occupying half of the data bit width of the write data interface. Using equal bit segmentation simplifies the hardware fragmentation logic and reduces timing pressure. For example, the physical channel and write data interface uniformly adopt a 512-bit standard bit width, with the lower 256 bits of the write data interface allocated entirely to the first write data interface unit, and the higher 256 bits allocated entirely to the second write data interface unit.

[0040] In other examples, the write data interface and physical channel can be configured to be segmented according to unequal bits, depending on the characteristics of the application instance of the computing system.

[0041] The pseudo-channel mapping module 1133 is configured to read the low-order consecutive bit data output by the first write data interface unit 1131_1 and map it to the first pseudo-channel PC0 of the corresponding physical channel; and to read the high-order consecutive bit data output by the second write data interface unit 1131_2 and map it to the second pseudo-channel PC1 of the corresponding physical channel. In this embodiment, as shown... Figure 4 As shown, the mapping logic is a static bit range binding, where the low-order write data flows to the first pseudo-channel PC0 and the high-order write data flows to the second pseudo-channel PC1, instead of as... Figure 2 The write data on the write data channel is interleaved and distributed to two pseudo channels.

[0042] With the mapping method of this embodiment, in narrow write scenarios such as 64-bit, 128-bit, and 256-bit, the write data on the write data channel will only need to occupy a single pseudo-channel, thereby avoiding the dual pseudo-channel RMW overhead triggered by narrow writes across channels in traditional interleaving mapping methods. Furthermore, when the number of bits of the narrow write data is equal to the data bit width of a single pseudo-channel, the RMW process can be completely eliminated, and only one write operation is performed, thereby significantly improving the transmission efficiency of narrow write operations.

[0043] In some examples, such as Figure 5As shown, the HBM controller 1130 can also be configured with a first pseudo-channel buffer 1134_1, a second pseudo-channel buffer 1134_2, and a scheduling module 1135. The pseudo-channel mapping module 1133 writes the low-order write data output by the first write data interface unit 1131_1 into the first pseudo-channel buffer 1134_1 for storage, and writes the high-order write data output by the second write data interface unit 1131_2 into the second pseudo-channel buffer 1134_2 for storage.

[0044] The HBM controller 1130 can be configured with a first pseudo-channel buffer 1134_1 and a second pseudo-channel buffer 1134_2 that are physically independent of each other; or it can be logically partitioned into a contiguous buffer space to form the first pseudo-channel buffer 1134_1 and the second pseudo-channel buffer 1134_2. This disclosure does not limit this. The scheduling module 1135 uses the amount of data transmitted in a single pseudo-channel as the scheduling granularity, reads the data buffered in the two buffers respectively, and sends them to the first pseudo-channel port and the second pseudo-channel port corresponding to the data transmission port for output. Specifically, the scheduling module 1135 sends the low-order write data in the first pseudo-channel buffer 1134_1 to the first pseudo-channel port of the corresponding data transmission port, and sends the high-order write data in the second pseudo-channel buffer 1134_2 to the second pseudo-channel port of the corresponding data transmission port.

[0045] As described above, one data transmission port corresponds to one physical channel, which is divided into a first pseudo-channel PC0 and a second pseudo-channel PC1. Correspondingly, the data transmission port is also divided into a first pseudo-channel port and a second pseudo-channel port. For example, the data transmission port includes 64 data pins, of which the first group of 32 data pins corresponds to the first pseudo-channel port, and the second group of 32 data pins corresponds to the second pseudo-channel port. The data transmission volume of a single pseudo-channel is 32 bits. The first pseudo-channel port transmits data in parallel in 8 steps within 4 clock cycles, splicing them together to form the first pseudo-channel PC0 with a width of 256 bits. The second pseudo-channel port transmits data in parallel in 8 steps within 4 clock cycles, splicing them together to form the second pseudo-channel PC1 with a width of 256 bits.

[0046] The HBM controller 1130 can be configured with an independent pseudo-channel buffer for each physical channel.

[0047] The HBM controller 1130 can be configured with an independent scheduling module for each physical channel. Each scheduling module operates independently and completes the data distribution scheduling of the pseudo-channels within its corresponding physical channel. Alternatively, the HBM controller 1130 can be configured with a common scheduling module, which centrally manages the data distribution tasks of all physical channels.

[0048] In these examples, by setting a buffer for each of the two pseudo channels of the physical channel, write data can be pipelined and buffered within the HBM controller. This decouples the timing of HBM internal scheduling from the data transmission on the on-chip bus, eliminates pipeline bubbles caused by the mismatch between the upstream data delivery rate and the HBM transmission rate, and improves continuous write bandwidth. <Second Embodiment> This disclosure also provides a data transmission mapping method for an HBM controller, which can be applied to an HBM controller according to a first embodiment. Figure 6 A flowchart illustrating the data transmission mapping method of this embodiment is shown. Figure 6 As shown, the method in this embodiment may include steps S610 to S630.

[0049] Step S610: Receive write data transmitted via the write data channel of the on-chip bus.

[0050] The HBM controller receives write data from the compute unit / cache via the write data channel on the on-chip bus through the write data interface 1131. The data width of the write data channel is equal to the data width of the write data interface 1131 and the data width of the physical channel. Corresponding to dividing the write data interface 1131 into the first write data interface unit 1131_1 and the second write data interface unit 1131_2, the write data channel is also divided into two consecutive bit segments from the least significant bit to the most significant bit. The least significant bit segment corresponds to the first bit segment of the first pseudo-channel PC0, and the most significant bit segment corresponds to the second bit segment of the second pseudo-channel PC1. Taking a write data channel with a data width of 512 bits as an example, the least significant 256 bits are the first bit segment, and the most significant 256 bits are the second bit segment.

[0051] The write data channel supports segmented valid masks, enabling the transmission of valid narrow write data, with invalid bits filled with irrelevant placeholder data. In this embodiment, write data refers to the valid bit data on the write data channel.

[0052] Step S620: Determine the bit field to which the received write data belongs.

[0053] The HBM controller parses the bit range of the current write data to determine whether the write data falls only in the first bit segment, only in the second bit segment, or is distributed across both bit segments.

[0054] Based on the HBM controller architecture of the first embodiment, the HBM controller can determine the bit segment corresponding to the current write data according to the distribution of the received write data within the write data interface. Specifically, when the complete write data is received only by the first write data interface unit, the write data is determined to belong to the first bit segment; when the complete write data is received only by the second write data interface unit, the write data is determined to belong to the second bit segment; when the write data is received by both the first and second write data interface units, it indicates that the write data includes a first data portion belonging to the first bit segment and a second data portion belonging to the second bit segment.

[0055] In step S630, if the write data belongs to a single bit segment, the write data is mapped to the single pseudo channel corresponding to the single bit segment for data transmission.

[0056] When it is determined that all the write data falls within a single bit segment, the write data is distributed to the corresponding pseudo-channel for data transmission, while the other pseudo-channel does not participate in this write operation. In some examples, if the write data belongs to a single bit segment and the number of bits in the write data is equal to the data width of the single bit segment, the complete write data can be directly mapped to the corresponding single pseudo-channel for data transmission without executing the RMW process. For example, if the write data channel is 512 bits, and a 256-bit low-order narrow write operation completely occupies the first bit segment, the HBM controller directly sends the 256 bits of write data to the first pseudo-channel PC0 for transmission without reading the original row data from the HBM. This eliminates the need for RMW read transactions and data replacement transactions, reduces the number of transactions in narrow write operations, and significantly improves the write efficiency of narrow write data.

[0057] In other examples, if the write data belongs to a single bit segment and the number of bits in the write data is less than the data width of the single bit segment, then the single pseudo-channel triggers the RMW process without the need for two pseudo-channels to trigger the RMW process. In this example, mapping the write data to the single pseudo-channel corresponding to the single bit segment for data transmission may include the following steps: reading the data block to which the write data belongs and matches the single pseudo-channel from HBM; replacing the corresponding fragment data in the data block with the write data based on the offset position of the write data within the data block; and mapping the complete data block after replacement to the single pseudo-channel and writing it back to HBM. In the HBM storage architecture, the basic storage unit that can be independently read and written is a storage row, also known as a storage page. The data bit width of a single storage row can be the same as the data bit width of the physical channel, for example, both being 512 bits. This storage row is divided into two parts according to the bit range: a first data block matching the first pseudo-channel and a second data block matching the second pseudo-channel. The data bit widths of the first data block and the second data block match the data bit widths of the first pseudo-channel and the second pseudo-channel, respectively, for example, 256 bits each.

[0058] For example, in a 128-bit low-order narrow write operation, where the write data only falls in the first segment, the HBM controller only reads the 256-bit first data block belonging to the write data and matching the first pseudo-channel PC0 to complete the modification. The 256-bit second data block corresponding to the second pseudo-channel PC1 remains untouched. In these examples, narrow write operations smaller than the data bit width of a single pseudo-channel only trigger the RMW process of that single pseudo-channel, without simultaneously occupying the resources of two channels, thus preserving the ability of the other channel to process other independent memory access requests in parallel.

[0059] In some other examples, if the written data contains both a first data portion belonging to the first bit segment and a second data portion belonging to the second bit segment, then the two data portions are mapped to the first pseudo-channel and the second pseudo-channel respectively for synchronous data transmission.

[0060] For each pseudo-channel, the HBM controller determines whether the RMW process needs to be performed on that pseudo-channel based on whether each data portion completely occupies the bit width range of the corresponding pseudo-channel.

[0061] For example, if the first data portion of this write operation completely fills the bit width range of the first pseudo-channel, while the second data portion does not fill the bit width range of the second pseudo-channel, the HBM controller directly maps the first data portion to the first pseudo-channel PC0 for transmission, without triggering the RMW process for the first pseudo-channel; at the same time, the RMW process is executed separately for the second data portion to complete the write update of the second data portion.

[0062] For example, if the first and second data portions of this write operation fill the bit width range of the first and second pseudo-channels respectively, i.e., the write data is 512 bits, then the HBM controller sends the data in parallel through the two pseudo-channels, utilizing the full bandwidth of the physical channel. In these examples, a complete wide write scenario can use two pseudo-channels in parallel without sacrificing full bandwidth transmission performance, simultaneously achieving narrow write optimization and full bit width transmission efficiency.

[0063] For scenarios where the AXI on-chip bus and write data channel have a data width of 512 bits, valid write data widths include 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, 256 bits, and 512 bits. Based on the address boundary alignment constraints of the AXI on-chip bus, the write data will simultaneously cover the first and second segments only when it completely occupies the 512-bit width, corresponding to occupying the first and second pseudo-channels. In this scenario, the bit range of both pseudo-channels is completely filled, allowing full bandwidth utilization of the entire physical channel without needing to perform a read-modify-write (RMW) process on any pseudo-channel. In other narrow write scenarios, the write data only occupies the first or second pseudo-channel, without simultaneously occupying both pseudo-channels.

[0064] In some examples, if the write data belongs to a single bit segment, while the write data is mapped to the single pseudo-channel corresponding to the single bit segment for data transmission, data from other memory access operations that are independent of this write operation can be transmitted through another pseudo-channel. Taking a 128-bit low-order narrow write scenario as an example, this narrow write data only occupies the first pseudo-channel PC0. The HBM controller can schedule the second pseudo-channel PC1 to synchronously transmit read data from independent read transactions or send write data from other independent write transactions. In these examples, the narrow write operation only occupies a single pseudo-channel, and the remaining pseudo-channels can be used to schedule other memory access transactions in parallel. This solves the resource waste problem of locking both pseudo-channels in a single narrow write under traditional interleaving mapping, and improves the efficiency of pseudo-channel resource reuse.

[0065] As can be seen from steps S610 to S630 above, this embodiment divides the write data channel into two consecutive bit segments for pseudo-channel mapping. In this way, the narrow write scenario will only occupy a single pseudo-channel, and either no RMW will be triggered, or only a single pseudo-channel will trigger RMW. At the same time, the idle pseudo-channel can process other memory access requests in parallel, which greatly reduces the overhead of read and write transaction switching in HBM narrow write scenario, fully releases the parallel transmission capability of dual pseudo-channels, and effectively improves the overall throughput performance of HBM write operation.

[0066] The mapping logic of the HBM controller in this embodiment can be adapted to segmented aligned storage architecture as well as traditional interleaved storage architecture.

[0067] For the segmented aligned storage architecture, consecutive low-order bit intervals within a storage row correspond to form the first data block adapted to the first pseudo-channel, and consecutive high-order bit intervals within a storage row correspond to form the second data block adapted to the second pseudo-channel. The storage row bit segmentation rules are completely consistent with the HBM controller's bit segmentation and pseudo-channel mapping rules for the write data channel.

[0068] In traditional interleaved storage architectures, data within a storage row is still divided into multiple data fragments with a fixed granularity (e.g., 32 bits). These fragments, belonging to the first pseudo-channel and the second pseudo-channel, are arranged alternately within the storage row. The HBM controller uses the exact same bit segment partitioning rules and pseudo-channel mapping logic for both read and write data channels, ensuring consistency in the bidirectional data mapping mechanism and guaranteeing strict alignment of bit data at both ends of the read and write operations. Specifically, during a write operation, HBM, based on the interleaved layout, distributes the continuous address write data from a single pseudo-channel into the storage row according to the data fragments. During a read operation, HBM sends the discretely distributed data fragments from the storage row back to the HBM controller through their corresponding single pseudo-channels. The HBM controller receives these data fragments and maps them to the corresponding continuous bit segments of the read data channel, completing the data fragment aggregation.

[0069] <Third Embodiment> This disclosure also provides a computing device, which includes multiple computing units and a high-bandwidth memory controller. The high-bandwidth memory controller interacts with the computing units via an on-chip bus. For example, the high-bandwidth memory controller may interact with the computing units via an on-chip cache. The high-bandwidth memory controller may have a microarchitecture according to a first embodiment and / or be used to execute a data transfer mapping method according to a second embodiment. The computing device is, for example,... Figure 1 The computing device 1100 in the middle.

[0070] <Fourth Embodiment> This disclosure also provides a computing system including high-bandwidth memory and a computing device. The computing device can be a computing device according to a third embodiment, or it can be configured to execute a data transfer mapping method according to a second embodiment. The high-bandwidth memory is connected to a high-bandwidth memory controller of the computing device to establish a physical channel for data interaction between the high-bandwidth memory and the high-bandwidth memory controller of the computing device.

[0071] The computing device of this computing system is, for example, Figure 1 The computing device 1100 in the middle, high bandwidth memory, for example is Figure 1 The high-bandwidth memory is 1300.

[0072] The computing system can be any type of electronic device, such as any type of chip, terminal device, workstation or server, etc., without any limitation.

[0073] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and apparatuses according to various embodiments of this specification. In this regard, each block in a flowchart or block diagram may represent a module, unit, or part of a circuit. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using hardware that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. Unless otherwise specified, implementation in hardware, implementation in software, and implementation using a combination of software and hardware can be equivalent.

[0075] Various embodiments of this specification have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A high-bandwidth memory controller, characterized in that, include: At least one data transmission port, each of the data transmission ports being configured to connect to high-bandwidth memory to establish a physical channel for data interaction between the high-bandwidth memory controller and the high-bandwidth memory; wherein the physical channel includes a first pseudo-channel and a second pseudo-channel; A write data interface configured to receive write data transmitted via the on-chip bus; wherein the data bit width of the write data interface is equal to the data bit width of the physical channel, and the write data interface is divided along the bit position from the least significant bit to the most significant bit into two consecutive bit interface units, namely a first write data interface unit and a second write data interface unit; and, A pseudo-channel mapping module is configured to map the low-order write data output by the first write data interface unit to a first pseudo-channel of the physical channel, and to map the high-order write data output by the second write data interface unit to a second pseudo-channel of the physical channel.

2. The high-bandwidth memory controller according to claim 1, characterized in that, It also includes a first pseudo-channel buffer, a second pseudo-channel buffer, and a scheduling module; The pseudo-channel mapping module is configured to: write the low-order write data output by the first write data interface unit into the first pseudo-channel buffer, and write the high-order write data output by the second write data interface unit into the second pseudo-channel buffer. The scheduling module is configured to use the amount of data transmitted in a single pseudo-channel as the scheduling unit, to send the low-order write data in the first pseudo-channel buffer to the first pseudo-channel port corresponding to the data transmission port, and to send the high-order write data in the second pseudo-channel buffer to the second pseudo-channel port corresponding to the data transmission port.

3. The high-bandwidth memory controller according to claim 1, characterized in that, The data width of both the physical channel and the write data interface is 512 bits; the lower 256 bits of the write data interface constitute the first write data interface unit, and the higher 256 bits constitute the second write data interface unit.

4. A data transfer mapping method for a high-bandwidth memory controller, characterized in that, The high-bandwidth memory controller is used to connect to high-bandwidth memory to establish a physical channel for data interaction between the high-bandwidth memory controller and the high-bandwidth memory, the physical channel including a first pseudo-channel and a second pseudo-channel; the method includes: Receive write data transmitted through the write data channel of the on-chip bus; wherein the data bit width of the write data channel is equal to the data bit width of the physical channel, and the write data channel is divided into two consecutive bit segments along the bit from low bit to high bit, namely the first bit segment corresponding to the first pseudo channel and the second bit segment corresponding to the second pseudo channel. Determine the bit field to which the written data belongs: If the write data belongs to a single bit segment, the write data is mapped to a single pseudo channel corresponding to the single bit segment for data transmission; wherein, the single bit segment is the first bit segment or the second bit segment.

5. The method according to claim 4, characterized in that, If the write data belongs to a single bit segment, the step of mapping the write data to the single pseudo-channel corresponding to the single bit segment for data transmission includes: When the number of bits in the write data is equal to the data width of the single bit segment, the write data itself is mapped to the single pseudo channel corresponding to the single bit segment for data transmission.

6. The method according to claim 4, characterized in that, If the write data belongs to a single bit segment, the step of mapping the write data to the single pseudo-channel corresponding to the single bit segment for data transmission includes: When the number of bits in the written data is less than the data width of the single bit segment, the complete data block to which the written data belongs and matches the single pseudo channel is read from the high-bandwidth memory; wherein, the number of bits in the data block is equal to the data width of the single bit segment; Based on the offset position of the written data within the data block, the corresponding data fragment in the data block is replaced by the written data. The data block that has been replaced is mapped to a single pseudo-channel corresponding to the single bit segment for data transmission, so as to write the complete data block that has been replaced back to the high-bandwidth memory.

7. The method according to any one of claims 4 to 6, characterized in that, After determining the bit field corresponding to the written data, the method further includes: If the written data includes a first data portion belonging to the first bit segment and a second data portion belonging to the second bit segment, then the first data portion is mapped to the first pseudo channel for data transmission, and the second data portion is mapped to the second pseudo channel for data transmission.

8. The method according to any one of claims 4 to 6, characterized in that, If the written data belongs to a single bit field, the method further includes: Schedule another pseudo-channel besides the single pseudo-channel to transmit data from other memory access operations that are independent of the write operation corresponding to the write data.

9. A high-bandwidth memory controller, characterized in that, The high-bandwidth memory controller is configured to perform the method according to any one of claims 4 to 8.

10. A computing device, characterized in that, include: Multiple computing units; as well as, A high-bandwidth memory controller, wherein the high-bandwidth memory controller is the high-bandwidth memory controller according to any one of claims 1 to 3 and 9, and the high-bandwidth memory controller interacts with the computing unit via an on-chip bus.

11. A computing system, characterized in that, include: The high-bandwidth memory and the computing device of claim 10, wherein the high-bandwidth memory is connected to the high-bandwidth memory controller of the computing device to establish a physical channel for data interaction between the high-bandwidth memory and the high-bandwidth memory controller of the computing device.

Images