Near-memory data reduction

Abstract

A method for implementing near-memory data reduction during memory operations to off-chip or off-die memory is provided. A near-memory reduction (NMR) unit (100) provides near-memory data reduction during write operations to a specified address range (110). The NMR unit is configured with an address range to be reduced, and when a memory operation specifies an address within the address range, the NRM unit performs data reduction by adding to an accumulated reduction result (120) a data value specified by the memory operation. According to one embodiment, the NRM unit maintains a count (130) of the number of updates to the accumulated reduction result, which is used to determine when data reduction is complete.

Description

Background Technology

[0001] The methods described in this section are pursued methods, but not necessarily methods previously conceived or pursued. Therefore, unless otherwise stated, no method described in this section should be assumed to be prior art solely because it is included in this section. Furthermore, no method described in this section should be assumed to be well-known, common, or conventional solely because it is included in this section.

[0002] Certain types of computational processing, such as machine learning, generate large datasets that must be reduced. Data reduction itself involves addition operations with a low computation-to-byte ratio, meaning few operations are performed on each byte of data read from memory. However, datasets larger than the available on-chip or die-based memory (such as cache) must be stored in main memory, and retrieving data from main memory back to on-chip or die-based memory to perform data reduction would be costly. The cost of data movement far outweighs the low computational cost of data reduction, and data movement can lead to cache pollution. The same problem applies to parallel computing methods.

[0003] One technical solution to the problem involves embedding data reduction operations onto the operations that generate the data to be reduced. While this allows data reduction to be performed while generating the data to be reduced, it has drawbacks. First, general-purpose computations (such as matrix multiplication and convolution) are typically implemented for reuse across various scenarios and optimally utilize available on-chip resources, such as CPU / GPU caches, GPU registers, LDS / shared memory, etc. Therefore, including data reduction operations can adversely affect performance and further lead to the creation and maintenance of even-numbered variables, thus hindering their reusability. Additionally, the operations that generate the dataset can employ specific techniques, such as multiple writers to the same output location, making it impractical to include data reduction operations within any single writer. This is typically done in a GPU using L2 atoms to merge the outputs from multiple writers into the same memory location. Therefore, a better method is needed for reducing the large amounts of data generated by computational operations. Attached Figure Description

[0004] In the accompanying drawings, embodiments are illustrated by way of example rather than limitation, and similar reference numerals refer to similar elements.

[0005] Figure 1 This is a block diagram illustrating a near-memory reduction (NMR) unit used to implement near-memory data reduction as described herein.

[0006] Figure 2 This is a flowchart illustrating a method for reducing data using NMR cells.

[0007] Figures 3A to 3EThis diagram illustrates an example implementation of a method for reducing data in the context of two memory modules.

[0008] Figure 4 This is a block diagram illustrating an example method for performing data reduction, which uses tree-based pairwise data reduction after each memory storage operation.

[0009] Figure 5 This is a block diagram illustrating the method used to perform "in-channel" data reduction.

[0010] Figure 6 This is a flowchart illustrating the method used to perform "in-channel" data reduction. Detailed Implementation

[0011] In the following description, numerous specific details are set forth for the purpose of explanation, to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments described are practiced without these specific details. In other instances, well-known structures and apparatuses are shown in block diagram form to avoid unnecessarily obscuring the embodiments.

[0012] I. Overview

[0013] II. Architecture

[0014] III. Near-Memory Data Reduction

[0015] A. Overview

[0016] B. Change the allocation of shared resources for threads

[0017] C. Usefulness indicators

[0018] D. Resolving resource allocation oscillations

[0019] IV. Operation Example

[0020] V. Intra-channel data reduction

[0021] I. Overview

[0022] A method is provided for implementing near-memory data reduction during memory operations to off-chip or off-die memory. A near-memory reduction (NMR) unit provides near-memory data reduction during write operations to a specified address range. The NMR unit is configured with an address range to be reduced, and when a memory operation is specified at an address within that address range, the NMR unit performs data reduction by adding the data value specified by the memory operation to an accumulated reduction result. In one embodiment, the NMR unit maintains a count of the number of updates to the accumulated reduction result, the count being used to determine when data reduction is complete.

[0023] The method avoids the cost of reading data from off-chip memory to on-chip memory to perform data reduction, and also stores the accumulated reduction result in an arithmetic unit or a register in the logic die of the microprocessor, stacked memory, or memory controller that is more easily accessible. Furthermore, the method reduces cache pollution and allows data reduction to be performed simultaneously with the computational operations that generate the data to be reduced, when the data is ready.

[0024] II. Architecture

[0025] Figure 1 This is a block diagram illustrating a Near Memory Reduction (NMR) unit 100 for implementing near memory data reduction as described herein. The NMR unit 100 includes a reduction address register 110, a result register 120, an optional counter register 130, an arithmetic logic unit (ALU) 140, and data reduction logic 150. Although embodiments are illustrated in the diagram and described herein in the context of an NMR unit 100 including ALU 140, in some embodiments, the NMR unit 100 includes digital circuitry for performing any type of data reduction operation (e.g., arithmetic and / or logical operations, such as addition, determining minimum or maximum values, AND, OR, XOR, etc.). The NMR unit 100 includes fewer or more elements, which vary depending on the specific implementation, and embodiments are not limited to any particular elements. According to one embodiment, the various registers described herein and illustrated as single registers are implemented by multiple registers, depending on the specific implementation.

[0026] The shrink address register 110 stores the address range for which data shrinking is performed. As described in more detail below, the stored address range is used by the data shrinking logic 150 to determine whether to shrink the physical or logical address range of the memory operation. The result register 120 stores the accumulated data shrinking result. In response to an initialization command, the result register 120 is initialized to a specified value, such as zero. Alternatively, in response to storing a new address range in the shrink address register 110, the result register 120 is initialized to a specified value.

[0027] Counter register 130 stores the current count of updates to result register 120 since its last initialization. For example, the memory controller uses this current update count to determine when data reduction is complete. Adder 140, implemented as a single adder or multiple adders, adds the data value used for the storage operation to the accumulated data reduction result stored in result register 120.

[0028] Data reduction logic 150 manages the data reduction performed by NMR unit 100. This includes identifying the memory operation to be reduced, performing data reduction, and optionally incrementing the current update count in counter register 130. The memory operation to be reduced specifies an address within the address range stored in reduction address register 110. To reduce the memory operation, NMR unit 100 causes the data value specified by the memory operation to be added to the accumulated data reduction result stored in result register 120. Data reduction logic 150 may also optionally increment the current update count stored in counter register 130. Data reduction logic 150 is implemented by computer hardware, computer software, or any combination of computer hardware and computer software.

[0029] According to one implementation, a set of primitives, i.e., primitive instructions or low-level instructions, are provided to allow processes to call functions on NMR unit 100. Example primitives include, but are not limited to:

[0030] program_reduction_address_range(address_begin,address_end)

[0031] read_reduction_result()

[0032] The `program_reduction_address_range(address_begin, address_end)` primitive prepares the NMR unit 100 for data reduction by programming the reduction address register 110 with the address range to be reduced. According to one embodiment, the address range is identified by a start address and a `length_count`, where the end address is calculated as `address_begin + length_count`. The address range is not limited to consecutive addresses, and according to one embodiment, the `program_reduction_address_range` primitive supports a stride parameter specifying the spacing between addresses to be reduced. Alternatively, the `program_reduction_address_range` primitive supports multiple address range parameters to implement data reduction over multiple non-consecutive, non-stepping address ranges. According to one embodiment, the `program_reduction_address_range` primitive initializes the accumulated data reduction result in the result register 120 and / or the current count in the counter register 130 to a specified value, such as zero. The specified value is included as a parameter of the `program_reduction_address_range` primitive or configured separately in the data reduction logic 150. Alternatively, separate primitives are provided for initializing result register 120 and / or counter register 130, such as `initialize_nmr(value)`, where the `value` parameter is the initialization value. The `read_reduction_result()` primitive causes the accumulated data to be read from result register 120 to reduce the result.

[0033] Although the implementation is illustrated in the figures and described herein in the context of address ranges and accumulated data reduction results being stored in registers in NMR unit 100, the implementation includes address ranges and accumulated data reduction results being mapped to memory locations accessible using conventional load and store operations.

[0034] According to one embodiment, NMR cell 100 is implemented as a separate entity assigned to a corresponding memory module. Alternatively, the various elements and functions described herein with respect to NMR cell 100 are implemented in a die of a memory controller or stacked memory package (such as a Hybrid Memory Cube (HMC) package or a High Bandwidth Memory (HBM) package). According to one embodiment, NMR cell 100 is implemented as a memory array / group close to a memory layer of 3D stacked memory or conventional DRAM, or close to or located in a memory controller or last-level cache. Any number of NMR cells 100 can be implemented to allow for simultaneous near-memory scaling. NMR cell 100 is used with any type of memory adaptable to near-memory computing, including, for example, non-volatile dual in-line memory modules (NVDIMMs) and NVDIMM-P, SRAM temporary storage, etc.

[0035] III. Near-Memory Data Reduction

[0036] Figure 2 This is a flowchart 200 illustrating a method for reducing data using NMR unit 100. In this example, addition is used to perform data reduction, but the implementation is not limited to this example and includes implementations using other types of data reduction operations. In step 202, initialization is performed, and initialization includes several different tasks that vary depending on the specific implementation. According to one implementation, NMR unit 100, more specifically result register 120 and / or counter register 130, is initialized to a specified value, such as zero. Other implementations use values ​​other than zero. According to one implementation, the initialization of NMR unit 100 is invoked by an initialization primitive or a program_reduction_address_range primitive that also specifies the address range to be reduced, as previously described herein. The address range to be reduced is the entire address range supported by the memory module or a subset of the address range supported by the memory module. According to one implementation, initialization includes initializing additional control information, such as the bit width and data type of the data to be reduced, for example, signed / unsigned integers, single / double precision floating-point numbers, etc. This allows the NMR unit 100 to correctly interpret the information received in the storage operation.

[0037] After initialization is complete, in step 204, a storage operation is received, specifying at least an address and a value, and a memory write to the address is performed. Depending on the specific implementation, the storage operation may specify additional information. According to one embodiment, the memory write is issued in a manner that bypasses or flows through the cache to the memory where near-memory reduction is performed. For example, in one embodiment, the storage operation includes parameters or is specifically designed to bypass the cache. In other embodiments, the value to be written to memory, along with additional metadata indicating which portions of the cache line have been written, is captured in the cache, but the value to be reduced is flushed from the cache to off-die memory to ensure that NMR unit 100 includes the data value in the data reduction. The metadata is tracked and passed to a reduction engine that indicates which subset(s) of the cache line have been modified and must be incorporated into the data reduction. This information is available in the form of a byte enable bit in architectures that support partial writes, such as some graphics processing units (GPUs) without error correction codes (ECC). NMR unit 100 receives the storage operation, for example, from a memory controller. Therefore, the method described in this paper is applicable to both cached storage operations and storage operations that bypass the cache and store data directly to off-chip memory.

[0038] In step 206, it is determined whether the storage operation is within the reduced address range. According to one embodiment, the address specified by the storage operation is compared with the address range stored in the reduced address register 110. For example, an address specified by the storage operation that is greater than or equal to the lower-order address and less than or equal to the higher-order address in the reduced address register 110 is within the reduced address range.

[0039] If it is determined in step 208 that the storage operation is within the reduced address range, then in step 210, the data value specified in the storage operation is added to the accumulated reduced result. For example, data reduction logic 150 causes the data value specified in the storage operation to be added to the accumulated data reduction result stored in the result register 120 via adder 140. Alternatively, the data value specified in the storage operation is processed or scaled by a function, for example, by a constant, to generate a processed data value, which is added to the accumulated data reduction result in the result register 120.

[0040] The current count stored in counter register 130 can optionally be incremented. For example, the current count, in integer form, is incremented by one each time data reduction is performed, i.e., when a storage operation is specified within the reduction address range. According to one embodiment, the current count in counter register 130 is used to stop data reduction, for example, after a specified number of iterations. For example, the algorithm uses the current count to provide approximate data reduction by limiting data reduction to a specified number of iterations within a specified address range, thereby reducing the consumption of computational resources.

[0041] The process is completed in step 214 after step 212, or in response to determining in step 208 that the storage operation is not within the reduced address range.

[0042] The above process is repeated an arbitrary number of times until data reduction is complete. The completion of data reduction is determined in different ways, depending on the specific implementation. For example, data reduction logic 150 performs data reduction for a specified number of iterations. As another example, the `program_reduction_address_range` primitive includes a parameter specifying the number of data reduction iterations to be performed. This allows the application to specify a specific number of data reductions to be performed. As yet another example, the memory controller or processing unit reads the current count stored in the counter register 130 of the memory module of interest and stops data reduction when a specified number of data reductions has been completed. For example, the memory controller retrieves the current count from the counter register 130 of multiple NMRs 100 and then notifies the microprocessor that data reduction is complete when all current counts meet a specified number. The microprocessor then reads the accumulated reduction results from the result register 120 of each NMR unit 100 and reduces these values ​​to the final reduction result. Alternatively, the final reduction result is generated by an arithmetic unit in the logic die of the stacked memory or near the memory controller.

[0043] Instead of reading the current count stored in counter register 130, the memory controller, knowing the address range to be reduced, tracks the number of memory operations to determine when data reduction is complete. According to one implementation, the completion of data reduction triggers tree reduction, as described in more detail below. In the implementation of cached data to be reduced, the completion of near-memory reduction includes triggering a flush of the cached data to off-chip memory to ensure that the data in the cache is included in the data reduction result.

[0044] A possible scenario is that NMR unit 100 receives a memory operation at a specified address that already contains a value from a previous memory operation before the data reduction cycle is considered complete. According to one embodiment, NMR unit 100 detects multiple writes to the same address and marks errors, for example, by setting an error bit, which can be polled and provided to the microprocessor by the memory controller. Alternatively, the new value is added to the accumulated data reduction result stored in result register 120, which is sufficient for fault-tolerant or approximation-based computations.

[0045] IV. Operation Example

[0046] Figures 3A to 3E An example implementation of a method for reducing data in the context of two memory modules (labeled memory module-0 and memory module-1 in the figure) is illustrated. Figures 3A to 3E The steps are illustrated in chronological order. In this example, the two memory modules have interleaved addresses; that is, addresses 0 and 2 are used for the first memory module and addresses 1 and 3 are used for the second memory module. This is typically used to provide memory module-level parallelism. Depending on the specific implementation, each memory module may have any number of other addresses. Each of the two memory modules has a corresponding NMR cell 100.

[0047] exist Figure 3A The process begins by initializing two memory modules. This includes setting the reduction value for each memory module to zero and specifying the address range to be reduced; in this example, the address range is addresses 0-3. As previously described herein, the `program_reduction_address_range` primitive is used to specify the address range to be reduced, and it also initializes result register 120 and counter register 130. Alternatively, result register 120 and counter register 130 are initialized using separate initialization commands. The address range to be reduced is stored in reduction address register 110.

[0048] exist Figure 3B In the first memory operation, the specified value 10 is to be stored at address 0. The memory controller managing the first memory module stores the value 10 at address 0 of memory module-0. Data reduction logic 150 of the NMR unit 100 for the first memory module determines whether the address specified in the memory operation is within the reduction address range stored in the reduction address register 110. Since the address 0 specified in the memory operation is within the reduction address range stored in the reduction address register 110, the data reduction logic 150 adds the value 10 specified in the memory operation to the accumulated reduction result stored in the result register 120 via adder 140. Figure 3BIn this context, this is illustrated by updating the reduced value -0 to 10, where the reduced value is the reduced value of the first memory module (i.e., memory module -0). The data reduction logic 150 of the NMR cell 100 of the first memory module also increments the current count stored in the counter register 130. The second memory module (i.e., memory module -1) remains unchanged because the second memory module does not process address 0.

[0049] If the address specified by the store operation is outside the reduced address range stored in the reduced address register 110, the value specified by the store operation will be stored in the first memory module, but the value will not be added to the accumulated reduced result stored in the result register 120. For example, suppose the store operation specifies that the value 7 will be stored at address 50. Assuming the first memory module is configured with address 50, the value 7 will be stored at address 50, but will not be added to the accumulated reduced result stored in the result register 120 because the address 50 specified in the store operation is not within the reduced address range specified in the reduced address register 110.

[0050] exist Figure 3C In the second memory operation, the specified value 99 is to be stored at address 1. The memory controller managing the second memory module causes the value 99 to be stored at address 1. The data reduction logic 150 of the NMR unit 100 for the second memory module determines whether the address specified in the memory operation is within the reduction address range stored in the reduction address register 110. Since the address 1 specified in the memory operation is within the reduction address range 0-3 stored in the reduction address register 110, the data reduction logic 150 causes the value 99 specified in the memory operation to be added to the accumulated reduction result stored in the result register 120 via adder 140. Figure 3C This is illustrated by updating the reduced value -1 to 99, where the reduced value is the reduced value of the second memory module (i.e., memory module-1). The data reduction logic 150 of the NMR cell 100 of the second memory module also increments the current count stored in the counter register 130. The first memory module (i.e., memory module-0) remains unchanged because the first memory module does not process address 1.

[0051] Continuing with the example, in Figure 3DIn the process, a third storage operation receives the specified value 44 and stores it at address 2. The memory controller managing the first memory module stores the value 44 at address 2. The data reduction logic 150 of the NMR unit 100 for the first memory module determines whether the address specified in the storage operation is within the reduction address range stored in the reduction address register 110. Since the address 2 specified in the storage operation is within the reduction address range 0-3 stored in the reduction address register 110, the data reduction logic 150 adds the value 44 specified in the storage operation to the accumulated reduction result stored in the result register 120 via adder 140. Figure 3D In this context, this is illustrated by updating the reduced value -0 to 54, where 54 is the sum of the previous value 10 and the new value 44. The data reduction logic 150 of the NMR cell 100 of the first memory module also increments the current count stored in the counter register 130. The second memory module (i.e., memory module -1) remains unchanged because the second memory module does not process address 2.

[0052] exist Figure 3E In the final operation illustrated, a fourth storage operation receives the specified value 22 and stores it at address 3. The memory controller managing the second memory module stores the value 22 at address 3. The data reduction logic 150 of the NMR unit 100 for the second memory module determines whether the address specified in the storage operation is within the reduction address range stored in the reduction address register 110. Since the address 3 specified in the storage operation is within the reduction address range 0-3 stored in the reduction address register 110, the data reduction logic 150 adds the value 22 specified in the storage operation to the accumulated reduction result stored in the result register 120 via adder 140. Figure 3E In this context, this is illustrated by updating the reduced value -1 to 121, where 121 is the sum of the previous value 99 and the new value 22. The data reduction logic 150 of the NMR cell 100 of the second memory module also increments the current count stored in the counter register 130. The first memory module (i.e., memory module -0) remains unchanged because the first memory module does not process address 3.

[0053] The technical solutions provided by these examples differ from those of in-memory atoms in that: 1) they provide memory reduction across a range of memory addresses rather than a single memory location; and 2) when in-memory atoms perform read-modify-write operations on memory locations (without memory bandwidth reduction), the solution stores the accumulated reduction results in the result register 120 of each NMR cell 100, which avoids the need to read data from main memory back to on-chip memory (e.g., cache) to perform data reduction.

[0054] V. Intra-channel data reduction

[0055] It is not uncommon for the width of a memory interface to exceed the width of the data elements being stored. For example, the logical width of a memory interface can be 256 bits or 512 bits, while the stored data elements can be 8 bits, 16 bits, 32 bits, or 64 bits wide. Therefore, for a memory module with a 256-bit wide interface, each memory storage operation will store eight 32-bit data elements.

[0056] One approach to performing data reduction simultaneously across multiple data elements in hardware is to use tree-based pairwise data reduction, or "cross-channel" data reduction. Figure 4 This is a block diagram illustrating an example method for performing tree-based pairwise data reduction after each memory store operation when the logical width of the memory interface exceeds the width of the stored data elements. In this example, the memory interface width is 256 bits, and each data element is represented by 32 bits, providing a maximum of eight 32-bit data elements per memory store operation. Figure 4 The width of the interface data element illustrated is an example value, and the implementation is not limited to these examples. Similarly, in this example, addition is used to perform data reduction, but the implementation is not limited to using addition to perform data reduction and includes other types of arithmetic and logical operations.

[0057] exist Figure 4 In the example, the data transfer process includes three memory store operations, with a full tree-based pairwise data reduction performed after each memory store operation. More specifically, after the first memory store operation, at the first level of the tree, four additions are performed on four pairs of 32-bit data elements to generate a pair sum. At the next level of the tree, two additions are performed on the four pair sums from the initial set of additions to generate two remaining pair sums. At the last level of the tree, the two remaining pair sums are added together to generate a result 1, which is the full data reduction of all eight 32-bit data elements from the first memory store operation, i.e., and .

[0058] The process is repeated for the eight 32-bit data elements in the second memory storage operation, and the data reduction result is added to result 1 to generate result 2. The eight 32-bit data elements from the third (and final) memory storage operation are reduced in a similar manner, and the result is added to result 2 to generate the final result, which is the sum of all 24 32-bit data elements from the three memory storage operations. For the three storage operations, this method requires at least three addition loops, where each addition loop requires seven additions and three steps to complete the data reduction, which is computationally cumbersome.

[0059] According to one implementation plan, with Figure 4In contrast to the cross-channel data reduction method, the intra-channel data reduction method performs addition within each 32-bit wide segment of the memory interface for all memory store operations across the data transfer. After the final memory store operation in the data transfer, a full tree-based pairwise data reduction is performed.

[0060] Figure 5 This is a block diagram illustrating a method for performing "in-channel" data reduction. Data is stored in memory using three memory storage operations, including two non-final memory storage operations and one final storage layer operation. According to one embodiment, each data channel is reduced individually across all memory storage operations. Figure 5 In the illustrated example, during each memory store operation, data reduction is performed on each 32-bit data element or channel, generating eight 32-bit reduced data values. For data reduction using addition, each 32-bit data element is added to the accumulated value of its corresponding 32-bit data element or channel, providing eight 32-bit accumulated data values. After the final memory store operation, a full tree-based pairwise data reduction is performed across the eight 32-bit reduced data values ​​to generate the final reduced data result. For data reduction using addition, the final reduced data result is the sum of all 32-bit data elements from all three memory store operations.

[0061] like Figure 5 As illustrated, assume the first memory store operation specifies a value of 17 for the first 32-bit data element. The value 17 is added to the accumulated value of the first 32-bit data element (i.e., the first channel). In this example, it is presumed that the initial accumulated value of the first 32-bit data value is zero, so after the first memory store operation, the accumulated value of the first channel is 17. This process is repeated for each of the other 32-bit data values ​​(i.e., channels). The accumulated data value is stored in a register, cache, main memory, or the NMR unit 100 previously described herein.

[0062] The second memory storage operation specifies the value 9 of the first 32-bit data element (i.e., the first channel). The value 9 is added to the accumulated value 17 of the first channel to generate a new accumulated value 26 for the first 32-bit data element. This process is repeated for each of the other 32-bit data values ​​(i.e., channels).

[0063] The third (and final) memory storage operation specifies the value 11 of the first 32-bit data element. Value 11 is added to the accumulated value 26 to generate a new accumulated value 37 for the first 32-bit data element. This process is repeated for each of the other 32-bit data values ​​(i.e., channels). After the third memory storage operation, each of the other eight 32-bit channels also has an accumulated value as the sum of the corresponding 32-bit data elements from each of the three memory storage operations. A full tree-based pairwise data reduction is then performed on the eight accumulated 32-bit data values ​​to generate… Figure 5 The final result is shown in the diagram.

[0064] This method is better than Figure 4 The method illustrated in the diagram is less computationally cumbersome because it performs a single set of parallel additions after each memory storage operation, instead of performing a full tree-based pairwise data reduction after each memory storage operation. Figure 5 In the illustrated example, a set of eight parallel additions is performed after each memory storage operation. Although the total number of additions is... Figure 4 The method for drawing in the middle is the same, but it requires fewer steps. Figure 4 In the example, a total of 23 additions and eleven steps are required, with each full tree-based pairwise data reduction requiring three steps, and two steps used to generate result 2 and the final result. In contrast, although Figure 5 The in-channel data reduction method also uses 23 additions, but only requires five steps.

[0065] Although the implementations are illustrated in the figures and described herein in the context of equal-width channels (i.e., all channels are 32-bit wide), the implementations are not limited to these examples. According to one implementation, the channels have different widths, and the memory controller or other element performing intra-channel data reduction is able to determine individual data elements within a memory storage operation, and the location of the determined data elements is consistent across all memory storage operations. The use of intra-channel data reduction is optional; for example, parameters of the memory storage operation and / or the memory controller are configured to perform intra-channel data reduction.

[0066] Figure 6 This is a flowchart 600 illustrating a method for performing "in-channel" data reduction. In step 602, initialization is performed, and this initialization includes several different tasks that vary depending on the specific implementation. According to one implementation, memory locations storing accumulated data values, such as registers, NRM unit 100, and other memory locations, are initialized to a specified value, such as zero. Depending on the specific implementation, other implementations use values ​​other than zero.

[0067] After initialization is complete, in step 604, a memory storage operation is received, which includes multiple data elements having a bit width smaller than the width of the memory interface. In step 606, the accumulated value of the data elements is updated. For example, the memory controller causes the value of the data element specified in the memory storage operation to be added to the accumulated value of each corresponding channel.

[0068] In step 608, it is determined whether the memory storage operation is a final memory storage operation. If not, control returns to step 604. If the current memory storage operation is a final memory storage operation, then in step 610, a full tree-based pairwise data reduction is performed to generate the final data reduction result, as previously described, and this process is completed in step 612.

Claims

1. A near-memory reduction unit, the near-memory reduction unit comprising: A reduced address register, configured to store a reduced address range. The result register, configured to store the data reduction result, and Data reduction logic, which is configured as follows: Determine whether the destination address for the storage operation used to store data in off-die memory is within the reduced address range stored in the reduced address register. In response to determining that the destination address for a storage operation to store data in the off-die memory is within the reduced address range stored in the reduced address register: The data values ​​of the storage operation are processed through a reduction operation to generate an updated data reduction result, and The updated data reduction result is stored in the result register.

2. The near-memory reduction unit of claim 1, wherein the data reduction logic is further configured to initialize the reduced address range to a value specified by an operation for programming the reduced address range in the near-memory reduction unit.

3. The near-memory reduction unit as described in claim 1, wherein the reduction address range is a range of contiguous memory addresses.

4. The near-memory reduction unit as described in claim 1, wherein the reduction address range is a step range of the memory address.

5. The near-memory reduction unit of claim 1, wherein in response to a change or initialization operation of the reduction address range stored in the reduction address register, the data reduction result stored in the result register is initialized to a specified value.

6. The near-memory reduction unit of claim 1, further comprising a counter register configured to store a current update count in the result register, wherein the data reduction logic is further configured to increment the current update count stored in the result register in response to determining that the destination address for the storage operation for storing data in the off-die memory is within the reduction address range stored in the reduction address register.

7. The near-memory reduction unit as claimed in claim 1, wherein the data reduction logic is further configured to: The data value of the storage operation used to store data in off-die memory is processed to produce a processed data value, and This causes the processed data value to be processed by the reduction operation to generate the updated reduction result.

8. The near-memory reduction unit of claim 7, wherein the processing of the data value includes one or more of processing the data value by a function or applying a constant to the data value.

9. The near-memory reduction unit as claimed in claim 1, wherein the off-die memory is main memory.

10. The near-memory reduction unit as claimed in claim 1, wherein: The data specified by the storage operation for storing data in off-die memory is stored in a cache, and Perform the storage operation to store the data specified by the storage operation from the cache to the main memory.

11. The near-memory reduction unit of claim 1, wherein the storage operation is a first storage operation, and the data reduction logic is further configured to set an error bit in response to receiving a second storage operation whose destination address is the same as the destination address of the first storage operation.

12. The near-memory reduction unit of claim 1, wherein the near-memory reduction unit is implemented in one or more of the following: a memory controller, a separate entity assigned to a memory module, or in a die of a stacked memory package.

13. The near-memory reduction unit as claimed in claim 1, wherein the reduction operation is an addition operation that adds the data value of the storage operation to the reduction result of the accumulated data stored in the result register.

Images