Systems for shuffle-accelerated, methods for processing data, storage nodes, and partitioning methods

Abstract

Systems for shuffle acceleration, methods for processing data, storage nodes, and partitioning methods are provided. A method for processing data in a system having a host and a storage node can include performing a shuffle operation on data stored at the storage node, where the shuffle operation can include performing a shuffle write operation and performing a shuffle read operation; where at least a portion of the shuffle operation is performed by an accelerator at the storage node. A method for partitioning data can include sampling, at a device, data from one or more partitions based on a number of samples; transferring the sampled data from the device to a host; determining, at the host, one or more characteristic partitioning criteria based on the sampled data; transferring the one or more characteristic partitioning criteria from the host to the device; and partitioning, at the device, data of the one or more partitions based on the one or more characteristic partitioning criteria.

Description

[0001] This application claims priority and benefit to U.S. Provisional Patent Application No. 63 / 116,144, filed November 19, 2020, entitled “Systems, Methods, and Devices for Storage Shuffle Acceleration,” and U.S. Application No. 17 / 112,975, filed December 4, 2020, which are incorporated herein by reference. Technical Field

[0002] This disclosure generally relates to data shuffle operations, and more specifically, to systems, methods, and apparatus for near-storage shuffle acceleration. Background Technology

[0003] Systems with a host and one or more storage nodes can utilize operations such as shuffling to rearrange data between partitions and / or nodes.

[0004] The information disclosed above in this background section is only intended to enhance the understanding of the background art of the invention, and therefore may contain information that does not constitute prior art. Summary of the Invention

[0005] A method for processing data in a system having a host and storage nodes may include: performing a shuffling operation on data stored at the storage nodes, wherein the shuffling operation may include: performing a shuffling write operation and performing a shuffling read operation; wherein at least a portion of the shuffling operation is performed by an accelerator at the storage nodes. The portion of the shuffling operation performed at the storage nodes may include a portion of the shuffling write operation. The portion of the shuffling write operation may include a partitioning operation. The portion of the shuffling write operation may include one or more of an aggregation operation, a sorting operation, a merging operation, a serialization operation, a compression operation, or an overflow operation. The portion of the shuffling operation performed at the storage nodes may include a portion of the shuffling read operation. The portion of the shuffling read operation may include one or more of an acquisition operation, a decompression operation, a deserialization operation, a merging operation, a sorting operation, or an aggregation operation. The portion of the shuffling operation performed at the storage nodes may include a partitioning operation performed using a peer-to-peer (P2P) connection between the storage device and the accelerator at the storage nodes. The portion of the shuffling operation performed at the storage nodes may include a data overflow operation performed using a P2P connection between the storage device and the accelerator at the storage nodes. The portion of the shuffling operation performed at the storage node may include an acquisition operation performed using direct memory access (DMI) operations. The portion of the shuffling operation performed at the storage node may also include a data merging operation performed using a P2P connection between the storage device at the storage node and the accelerator.

[0006] A storage node may include: a storage device; and an accelerator; wherein the storage node is configured to use the accelerator to perform at least a portion of a shuffling operation. The storage node may also include a P2P connection between the storage device and the accelerator; and the storage device and the accelerator may be configured to perform said portion of the shuffling operation by transmitting data via the P2P connection. The accelerator may be integrated with the storage device. The storage node may include a server. The storage device may be a first storage device, the accelerator may be a first accelerator, the P2P connection may be a first P2P connection, and the storage node may further include: a second storage device, a second accelerator, and a second P2P connection between the second storage device and the second accelerator; wherein the second storage device and the second accelerator may be configured to perform said portion of the shuffling operation by transmitting data via the second P2P connection. The first storage device and the second storage device may be configured to perform said portion of the shuffling operation by transmitting data via direct memory access operations. The storage node may be configured to perform said portion of the shuffling operation by transmitting data to an additional storage node via remote direct memory access operations.

[0007] A method for partitioning data may include: sampling data from one or more partitions at a device based on a number of samples; transmitting the sampled data from the device to a host; determining one or more feature partitioning criteria at the host based on the sampled data; transmitting the one or more feature partitioning criteria from the host to the device; and partitioning the data in the one or more partitions at the device based on the one or more feature partitioning criteria. The method may further include: determining the number of records in the one or more partitions at the device; transmitting the number of records in the one or more partitions from the device to the host; determining the number of samples in the one or more partitions at the host; and transmitting the number of samples from the host to the device. The sampling may be performed by an accelerator at the device.

[0008] A system may include: a storage node including an accelerator; and a host configured to perform a shuffling operation on data stored at the storage node, wherein the shuffling operation may include a shuffling write operation and a shuffling read operation; and the storage node may be configured to use the accelerator to perform at least a portion of the shuffling operation. Attached Figure Description

[0009] The accompanying drawings are not necessarily drawn to scale, and throughout the drawings, elements with similar structures or functions may generally be indicated by similar reference numerals or portions thereof for illustrative purposes. The drawings are intended only to facilitate the description of the various embodiments described herein. The drawings do not depict every aspect of the teachings disclosed herein and do not limit the scope of the claims. To prevent the drawings from becoming unclear, not all components, connections, etc., may be shown, and not all components may have reference numerals. However, the pattern of component configuration may be readily apparent from the drawings. The drawings, together with the specification, illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0010] Figure 1 An example of a shuffling operation for a distributed data processing framework is shown.

[0011] Figure 2 An example of a shuffling architecture for a distributed data processing framework is shown.

[0012] Figure 3 An embodiment of a system for accelerating rinsing according to the present disclosure is shown.

[0013] Figure 4 An embodiment of a mixing and rinsing acceleration system with P2P connectivity according to the present disclosure is shown.

[0014] Figure 5 An example embodiment of a shuffling acceleration system with P2P connectivity according to the present disclosure is shown.

[0015] Figure 6 Another example embodiment of a shuffling acceleration system with logical P2P connectivity via a switch, according to this disclosure, is shown.

[0016] Figure 7 Another example embodiment of a shuffling acceleration system according to this disclosure, having a logical P2P connection via a switch integrated with the accelerator, is shown.

[0017] Figure 8 This illustration shows a PCIe base address register mapping that can be used to implement the switching function of a PCIe switch according to the present disclosure.

[0018] Figure 9 An embodiment of a wash architecture with accelerated wash according to the present disclosure is shown.

[0019] Figure 10 An embodiment of a shuffling acceleration system with P2P connectivity and a DMA / RDMA engine according to the present disclosure is shown.

[0020] Figure 11 An embodiment of a method for processing data in a system having a host and storage nodes according to the present disclosure is shown.

[0021] Figure 12 An example of a partition operation for a shuffle write operation is shown.

[0022] Figure 13 An example embodiment of a partitioned workflow architecture according to this disclosure is shown. Detailed Implementation

[0023] Overview

[0024] Some inventive principles of this disclosure relate to offloading one or more portions of a shuffle operation from the host to a storage node. For example, one or more portions of a shuffle write operation and / or a shuffle read operation may be performed by an accelerator at the storage node. According to the implementation details, this can reduce the processing workload at the host and / or reduce input and / or output (I / O) operations and / or network transfers between the host and one or more components of one or more storage nodes.

[0025] Some additional inventive principles of this disclosure relate to using a peer-to-peer (P2P) connection between a storage device and an accelerator for one or more portions of a shuffling operation. For example, a P2P connection can transfer data between the storage device and the accelerator for one or more portions of a shuffling write operation and / or a shuffling read operation. According to the implementation details, this can reduce I / O operations and / or network transfers between the host and one or more components of one or more storage nodes.

[0026] Some additional inventive principles of this disclosure relate to using one or more types of direct memory access (DMA) operations for one or more portions of a shuffling operation. For example, DMA operations or remote DMA (RDMA) operations can be used to transfer data between storage devices within a node or between different nodes. According to the implementation details, this can reduce I / O operations and / or network transfers between storage devices between nodes and / or within a node.

[0027] The principles disclosed herein may have independent utility and may be implemented individually, and not every principle can be utilized in every embodiment. Furthermore, the principles may be implemented in various combinations, some of which may synergistically enhance the benefits of the individual principles.

[0028] Mixed washing operation

[0029] In some embodiments, shuffling operations can be used to rearrange data among partitions, devices, and / or nodes in a distributed data processing framework. This can be helpful, for example, when the transformation involves data from other partitions (such as sums in columns).

[0030] Figure 1 An embodiment of a shuffling operation for a distributed data processing framework is illustrated. The shuffling operation 102 may include at least two parts: a map task and a reduce task. One or more map tasks may receive, for example, chunks of collocated data 104A, 104B, and 104C (collectively, 104) from one or more prior processors (such as map processors and / or filter processors). The map tasks may partition the received data into different blocks 106A, 106B, and 106C (collectively, 106) based on one or more partitioning rules (such as range partitioning and / or list partitioning). Figure 1 In the embodiment shown, the partitioning rule can be implemented as range partitioning 108. Therefore, in this embodiment, numerical inputs can be assigned to different partitions, which can be indicated by different types of shading in portions of block 106. The mapping task can then concatenate the different blocks 106 to a file and write (e.g., persist) the file to local storage as intermediate mapping output.

[0031] In the reduction portion of shuffling operation 102, one or more reduction tasks may request (e.g., read) blocks from intermediate mapping outputs to form a specific partition. Therefore, as... Figure 1As shown by the different shaded areas, entries from different input data blocks that have been assigned to the same partition can be gathered into the same reduced output block 110A, 110B, or 110C (collectively referred to as 110), which can now coextensify with the partition. After shuffling operation 102, the data may have been swapped, allowing one or more post-shuffling operations (e.g., sorting) to continue to be performed on one or more of the reduced-side partitions 110. In some embodiments, such as after one or more sorting operations 114, shuffling operation 102 may ensure the partition order and / or indirectly ensure or guarantee the global order of the dataset, which may include blocks 112A, 112B, and 112C (collectively referred to as 112).

[0032] although Figure 1 The embodiment shown may be illustrated with three input data blocks 104, three mapped output blocks 106, and three reduction output blocks (e.g., partitions) 110, but any number of blocks may be used at any stage of the shuffling operation 102. For example, in some embodiments, if the range partitioning rule specifies 100 ranges, then there may be 100 reduction output blocks 110.

[0033] In some embodiments, the shuffle operation 102 may be implemented using at least a shuffle write operation and a shuffle read operation. For example, the shuffle write operation may be performed by a mapping task that rearranges input data into one or more blocks that may include entries belonging to different partitions. The shuffle write operation may write these blocks to local storage as intermediate mapping output. For example, the shuffle read operation may be performed by a reduction task that obtains the mapping state that has been logged to the drive by the shuffle write operation. The mapping state may indicate which blocks of the intermediate mapping output may contain data entries for each partition. The reduction task may fetch one or more blocks or portions of blocks of the intermediate output based on the mapping state. If any blocks reside on different nodes, the reduction task may fetch those blocks or portions of those blocks over the network.

[0034] Figure 2 An example of a shuffling architecture for a distributed data processing framework is shown. Figure 2 In the embodiments shown, mapping task 202 (also referred to as a mapper) is generally shown on the left side of the figure, and reduction task 204 (also referred to as a reducer) is generally shown on the right side of the figure, and conceptually separated by a shading bar running through the middle of the figure. In some embodiments, substantially all processing operations (e.g., computations) may be performed by a central processing unit (CPU).

[0035] As indicated by arrow 207, the shuffling operation can begin when the action in execution method 206 of reduction task 204 triggers initiation operation 208 of mapping task 202. Initiation operation 208 can initiate shuffling write operation 210 using input data and / or shuffling dependency information.

[0036] The shuffle write operation 210 may include a data partitioning operation 212, in which the CPU uses one or more I / O operations via a Peripheral Component Interconnect Fast (PCIe) interconnect to retrieve data from one or more storage devices. The data partitioning operation 212 may then partition the input data into one or more blocks by assigning a partition identifier (ID) to each entry of the input data according to partitioning rules.

[0037] Data merge operation 214 can merge data entries belonging to the same partition (e.g., data with the same partition ID) into contiguous blocks of mapped output data. Data merge operation 214 can also, for example, sort and / or aggregate data according to one or more shuffling requirements.

[0038] When the amount of data being processed reaches an overflow threshold, shuffle write operation 210 may initiate a dataspill sequence. In some embodiments, the dataspill sequence may include data serialization and / or data compression operations 216, which may reduce the amount of mapped output data transferred via I / O operations. Then, in dataspill operation 218, the CPU may use one or more I / O operations to write the mapped output data to one or more files in local storage. In operation 220, the CPU may register mapped state data with a driver for a distributed data processing framework, the mapped state data including metadata of the mapped output data. The driver may publish the mapped state data for use throughout the framework.

[0039] The execution method 206 in reduction task 204 can also initiate a shuffle read operation 222, in which the CPU may request a mapping state from the driver in operation 224. During the acquisition operation 226, the CPU can then use the mapping state to acquire one or more blocks, for example, for each partition. If the CPU and the input data are located on different nodes, the CPU can acquire the data via a network and / or network protocols (such as Ethernet and / or Transmission Control Protocol / Internet Protocol (TCP / IP)). In some embodiments, the shuffle read operation 222 may include a data decompression and / or data deserialization operation 228, in which the CPU may convert the received data back to its original form.

[0040] During data merging operation 230, the CPU may merge data entries belonging to the same partition into contiguous blocks of reduced output data, which the CPU may then write to local storage via one or more I / O operations. In some embodiments, data merging operation 230 may also sort and / or aggregate the data, for example, according to one or more shuffling requirements. Reduce task 204 may then proceed with one or more post-shuffling operations 232 (such as sorting operations).

[0041] like Figure 1 and Figure 2 As shown, in some embodiments, the shuffling operation may involve one or more CPU-intensive operations (such as serialization / deserialization, compression / decompression, sorting, and / or merging operations). The shuffling operation may also involve a large number of I / O operations and / or network transfers of many small fragmented files. Furthermore, for example, when the data is too large to fit in memory, partitioning operations (such as operation 212) may involve additional CPU-intensive sampling operations and / or additional segmented I / O operations. Additionally, when data is written to and / or read from storage devices and / or transferred over the network during the shuffling operation, it can halt the distributed data processing framework's ability to perform other processing. Therefore, the shuffling operation can lead to performance bottlenecks, especially in big data, shuffling-intensive applications. Furthermore, the shuffling operation can put pressure on the CPU, memory, storage devices, and / or network capacity of one or more devices and / or clusters running the distributed data processing framework.

[0042] In some embodiments, reducing the number of shuffling operations performed or the amount of data transferred during shuffling operations can be beneficial. However, shuffling data over a network in a many-to-many fashion can be meaningful. In some embodiments, all or most of the entire working set (which may be a large portion of the input data) can be transferred over the network. This can, for example, place a significant burden on the operating systems (OS) at the source and / or destination by requiring numerous file and / or network I / O operations.

[0043] Mixed washing speeds up

[0044] In some embodiments according to this disclosure, one or more portions of the shuffling operation may be offloaded to an accelerator at the storage node.

[0045] Figure 3 An embodiment of a system for accelerating rinsing according to the present disclosure is shown. Figure 3The embodiments shown may include a CPU 302, a solid-state drive (SSD) 304, a field-programmable gate array (FPGA) 306, dynamic random access memory (DRAM) 308, a PCIe switch 310, and PCIe links 312, 314, and 316. Figure 3 The PCIe topology shown can be configured such that the SSD 304 and FPGA 306 act as endpoints of the host CPU 302. Therefore, the CPU 302 can be responsible for transferring data between the SSD 304 and FPGA 306, and between the SSD 304 and the DRAM 308 used by the FPGA 306.

[0046] if Figure 3 The system shown is used to implement mixed washing operations (such as, Figure 2 As shown in the shuffle operation, one or more processing portions of shuffle write operation 210 and / or shuffle read operation 222 can be offloaded to FPGA 306. For example, one or more portions of partition operation 212, merge operations 214 and 230, serialization and / or compression operation 216 / deserialization and / or decompression operation 228, data overflow operation 218, and / or fetch operation 226 can be offloaded to FPGA 306. Depending on the implementation details, offloading one or more portions of these operations can reduce the clock cycles consumed by CPU 302. However, offloading operations to FPGA 306 can also involve transferring data to and from FPGA 306 and DRAM 308 via CPU 302. This can increase communication costs between components, which can impose latency and / or throughput limitations on the system. Furthermore, if multiple SSDs and / or FPGAs are added to the system, the CPU 302 may lack the ability to scale its communication with the SSDs and / or FPGAs due to high CPU overhead and / or limited PCIe bandwidth.

[0047] Some embodiments of this disclosure may include a P2P connection between one or more storage devices and one or more accelerators, the P2P connection being implemented as a private connection.

[0048] Figure 4 An embodiment of a mixing and rinsing acceleration system with P2P connectivity according to the present disclosure is shown. Figure 4 The embodiments shown may include a host 402, a storage node 404, and a communication interface 406. The storage node 404 may include a storage device 408, an accelerator 410, and a P2P connection 412.

[0049] P2P connections (such as, Figure 4The use of a P2P connection (as shown in the diagram) allows the accelerator to directly access data in one or more storage devices, thereby saving limited bandwidth on connections between the CPU and storage devices, as well as connections between the CPU and the accelerator. According to implementation details, P2P connections can increase bandwidth and / or reduce overhead, memory usage, and / or power consumption associated with transferring data between storage devices and the accelerator compared to transferring data via the host and / or host memory. In some embodiments, P2P connections can be particularly helpful for shuffling acceleration operations, which may involve multiple data migrations between the accelerator and storage devices.

[0050] In some embodiments, depending on the details of the implementation, performing computation at or near the storage device (e.g., through the use of an accelerator) can reduce the cost and / or power consumption of I / O operations. This can also increase system scalability, for example, in the context of managing a large storage system. However, scalability in a large storage system with multiple storage devices (such as SSDs) can be limited by, for example, CPU overhead related to reading and / or writing data and / or shared connection bandwidth and / or the capacity of the host memory. In some embodiments, depending on the details of the implementation, shuffling acceleration techniques using P2P communication, as disclosed herein, can increase system scalability by removing or mitigating one or more of these bottlenecks.

[0051] Refer again Figure 4 The communication interface 406 provides a first communication connection between the host 402 and the storage device 408, and a second communication connection between the host 402 and the accelerator 410. In some embodiments, the first and second communication connections can be implemented through separate physical and logical connections. In some embodiments, the first and second communication connections can be implemented through physical and / or logical connections that can be partially or completely combined. For example, the storage node 404 may include a physical switch that can implement the first and second communication connections as partially separate physical connections. As another example, the first and second communication connections can be implemented as separate logical or virtual connections on a combined physical connection.

[0052] Communication interface 406 can be implemented using any type of communication architecture and / or protocol. For example, communication interface 406 can be implemented entirely or partially using interconnect architectures and / or protocols (such as PCIe, Fast Compute Link (CXL), and / or Cache Coherent Interconnect for Accelerators (CCIX), etc.). As another example, communication interface 406 can be implemented entirely or partially using network architectures and / or protocols (such as Ethernet, TCP / IP, Fibre Channel, and / or Unlimited Bandwidth, etc.). As another example, communication interface 406 can be implemented entirely or partially using storage interfaces and / or protocols (such as Serial ATA (SATA), Serial Attached SCSI (SAS), and / or Non-Volatile Memory Fast (NVMe), etc.). Furthermore, any of these architectures, protocols, and / or interfaces can be combined in hybrid combinations (such as NVMe over a network (NVMe-oF)).

[0053] The P2P connection 412 can be implemented using any type of communication architecture and / or protocol (such as the interconnect, network, and / or storage interfaces described above). In some embodiments, the P2P connection 412 can be implemented entirely or partially as a separate logical or virtual connection over a shared physical connection that can be used to implement the communication interface 406.

[0054] Host 402 can be implemented using any type of processing device. Examples may include one or more general-purpose or special-purpose CPUs, as well as FPGAs, application-specific integrated circuits (ASICs), systems-on-a-chip (SoCs), and / or any other components that can perform the functions of a host processor for a distributed data processing framework (such as Apache Spark and / or Apache Hadoop, etc.). General-purpose or special-purpose CPUs include complex instruction set computer (CISC) processors and / or reduced instruction set computer (RISC) processors, etc.

[0055] Storage device 408 can be implemented using any type of storage device, such as hard disk drive (HDD), SSD and / or persistent storage device (such as cross-grid memory with volume resistance variation).

[0056] Accelerator 410 can be implemented using any type of processing device, including one or more CISC and / or RISC processors, FPGAs, ASICs, SOCs and / or graphics processing units (GPUs), and any combinational logic, sequential logic, timers, counters, registers, gate arrays, complex programmable logic devices (CPLDs), state machines, etc. In some embodiments, the accelerator may be implemented as part of the memory controller of memory device 408. In some embodiments, one or more memories (such as DRAM) may be provided for or integrated with accelerator 410 to provide workspace memory for one or more portions of a shuffling operation that can be offloaded to accelerator 410.

[0057] In some embodiments, accelerator 410 may implement some or all of the offloading shuffling operations primarily in software, such as running on a general-purpose or special-purpose CPU. In some embodiments, accelerator 410 may implement some or all of the offloading shuffling operations primarily in hardware. For example, one or more offloading shuffling operations may be implemented with dedicated logic on an ASIC. As another example, one or more offloading shuffling operations may be programmed into an FPGA. Depending on the implementation details, hardware implementation of the offloading shuffling operations may provide increased throughput, reduced latency, reduced memory usage, and / or reduced power consumption.

[0058] Although host 402 is shown as a separate component, host 402 may be implemented integrated with storage node 404. Similarly, although storage device 408 and / or accelerator 410 are shown as integrated with storage node 404, storage device 408 and / or accelerator 410 may be implemented independently of storage node 404. In some embodiments, accelerator 410 may be integrated with storage device 408.

[0059] Figure 4 The embodiments shown can be implemented in any physical configuration. For example, in some embodiments, the system can be implemented as a server chassis, where the host 402, storage device 408, and accelerator 410 are implemented as separate components, and the communication interface 406 and P2P connection 412 can be implemented via PCIe links through a backplane and / or midplane, etc. In such a configuration, the first communication connection between the host 402 and the storage device 408 can be implemented as a first point-to-point PCIe link, the second communication connection between the host 402 and the accelerator 410 can be implemented as a second point-to-point PCIe link, and the P2P connection 412 can be implemented as a third point-to-point PCIe link. Optionally, one or more of the point-to-point PCIe links can be combined using one or more PCIe switches.

[0060] In another example physical configuration, storage node 404 may be implemented as a server chassis containing storage device 408 and accelerator 410, while host 402 may be implemented in a separate chassis or rack, or in a remote location. In one example, accelerator 410 is integrated with storage device 408, and storage node 404 includes a server. In this configuration, communication interface 406 may be implemented using network architectures and / or protocols (such as Ethernet and TCP / IP), and storage device 408 may be implemented as an Ethernet SSD (eSSD). Additionally, in this configuration, network switches may be provided on backplanes, midplanes, and / or switch boards to provide connectivity between storage device 408 and host 402 and / or between accelerator 410 and host 402. In this configuration, P2P connection 412 may be implemented, for example, via point-to-point PCIe, or via a PCIe switch on backplanes, midplanes, and / or switch boards. Optionally or additionally, P2P connection may be implemented as a logical connection via network switches as described above.

[0061] Figure 5 An example embodiment of a shuffling acceleration system with P2P connectivity according to the present disclosure is shown. Figure 5 The embodiments shown may include a host 502, a storage node 504, a communication interface 506, a storage device 508, an accelerator 510, and a P2P connection 512, which may be similar to Figure 4 The hosts, storage nodes, communication interfaces, storage devices, accelerators, and P2P connections shown can be implemented using any of the implementation details disclosed above. However, Figure 5 The embodiments shown may include a switch 514, which enables a first communication connection 516 between the host 502 and the storage device 508 and a second communication connection 518 between the host 502 and the accelerator 510. In some embodiments, the switch 514 may be implemented as a network switch (such as an Ethernet switch) and / or an interconnect switch (such as a PCIe switch), etc.

[0062] Figure 6 Another example embodiment of a shuffling acceleration system with logical P2P connectivity via a switch, according to this disclosure, is shown. Figure 6 The embodiments shown may include a host 602, a storage node 604, a communication interface 606, a storage device 608, an accelerator 610, and a switch 614. The host 602, storage node 604, communication interface 606, storage device 608, accelerator 610, and switch 614 may be similar to Figure 4 and Figure 5The hosts, storage nodes, communication interfaces, storage devices, accelerators, and P2P connections shown can be implemented using any of the implementation details disclosed above. However, in Figure 6 In the embodiment shown, the P2P connection 612 between the storage device 608 and the accelerator 610 can be implemented as a logical connection via a switch 614.

[0063] Figure 7 Another example embodiment of a shuffling acceleration system according to this disclosure, having a logical P2P connection via a switch integrated with the accelerator, is shown. For illustrative purposes, Figure 7 The embodiments shown may include some specific implementation details (such as PCIe interconnect topologies, FPGA accelerators, and SSD storage devices). However, in other embodiments, these components may be replaced by alternatives (such as HDDs and / or storage drives based on persistent cross-grid memory with varying bulk resistance, interconnects or network interfaces (such as CXL and / or Ethernet), and ASICs and / or SOCs).

[0064] exist Figure 7 In the embodiment shown, the storage node may be implemented as an enhanced SSD 702, which has an SSD controller 704, NAND flash memory media 706, an FPGA 708, and DRAM 710. The FPGA 708 may include a PCIe switch 712 and an accelerator 714. The PCIe switch 712 may be physically connected to the CPU (host) 716, the SSD controller 704, and the accelerator 714 via PCIe links 718, 720, and 722, respectively. However, the switch 712 may implement logical or virtual connections 724, 726, and 728. Logical connection 724 may handle data read and / or write transfers between the SSD controller 704 (which may appear as a separate endpoint) and the host 716. Logical connection 726 may handle data read and / or write transfers between the accelerator 714 (which may appear as a separate endpoint) and the host 716. Logical connection 728 can handle P2P communication between SSD controller 704 and accelerator 714. In some embodiments, one or more switching functions can be implemented as softswitch functions.

[0065] For convenience and / or availability of the integrated circuit (IC) area, the PCIe switch 712 may be physically integrated into the FPGA 708. However, in other embodiments, the PCIe switch 712 may be a separate component or may be integrated into the SSD controller 704. In other embodiments, Figure 7 Any number of components shown can be integrated onto a SOC.

[0066] Figure 8This diagram illustrates a PCIe Base Address Register (BAR) mapping that can be used to implement the switching functionality of PCIe switch 712. In address mapping 802, NVMe register address range 804 supports NVMe transfers between SSD controller 704 and host 716. Accelerator address range 806 and DRAM address range 808 support data transfers between host 716 and accelerator 714, and between host 716 and DRAM 710. In some embodiments, DRAM used for the FPGA may be exposed to the host PCIe address space. In some embodiments, NVMe commands can securely stream data between SSD controller 704 and FPGA 708 using P2P connection 728.

[0067] Figure 7 The embodiments shown (which can be used to implement any shuffling acceleration techniques disclosed herein) can provide enhanced concurrency, and in some implementations can provide unlimited concurrency. In some embodiments, depending on the implementation details, this configuration can: save cache (e.g., L2: DRAM) bandwidth; achieve scaling without requiring expensive CPUs at the storage nodes; and / or avoid data movement to and / or from separate accelerators and / or from their funnels.

[0068] Figure 9 An embodiment of a wash architecture with accelerated wash according to the present disclosure is shown. Figure 9 The embodiments shown may include a mapping task 902 and a reduction task 904, wherein the mapping task 902 includes a shuffle write operation 910 and the reduction task 904 includes a shuffle read operation 922. These tasks and shuffle operations may include similar... Figure 2 Some of the elements shown may have similar designations and may operate in a similar manner. However, in Figure 9In the system illustrated, one or more portions of a shuffling operation (e.g., CPU-intensive operations) may be offloaded to an accelerator. In some embodiments, all or part of any operation shown in shaded areas may be offloaded to an accelerator, while operations not shown in shaded areas may be performed by the CPU. For example, in a shuffling write operation 910, all or part of a data partitioning operation 912, a data merging operation (which may include data sorting and / or data aggregation operations) 914, a data serialization and / or data compression operation 916, and / or a data overflow operation 918 may be offloaded to an accelerator. As another example, in a shuffling read operation 922, all or part of a get operation 926, a data decompression and / or data deserialization operation 928, and / or a data merging operation (which may include data sorting and / or data aggregation operations) 930 may be offloaded to an accelerator. Examples of serialization operations may include converting an object or other data into a bitstream or bytestream for transmission to a storage device. Examples of deserialization operations may include converting a bitstream or bytestream back into an object or other data after transmission from a storage device. Serialization and / or deserialization operations can be performed, for example, using serialization / deserialization features in Java and / or Kryo, etc. Figure 9 In the embodiments shown herein, and in any other embodiments disclosed herein, elements are marked as optional without indicating that other elements are mandatory.

[0069] Figure 9 The embodiments shown can be used with any acceleration architecture according to this disclosure (including...). Figures 3 to 8 (The acceleration architectures shown in the diagram) can be used for implementation. If implemented using a system with P2P communication between the storage device and the accelerator, any or all of partitioning operation 912, data overflow operation 918, and / or data merging operation 930 can utilize a P2P connection. For example, data merging operation 930 can directly write intermediate mapped output data to local storage (such as an SSD) via a P2P connection instead of using relatively expensive I / O operations with the CPU. As another example, partitioning operation 912 can use a P2P connection to write data to local storage, as described below. Depending on the implementation details, the use of a P2P connection can increase throughput and / or reduce cost, latency, memory usage, and / or power consumption compared to using relatively expensive I / O operations with the CPU.

[0070] Figure 9The embodiments shown may also utilize one or more Direct Memory Access (DMA) technologies. For example, if the data resides within the same node (e.g., in the same storage device, or in storage devices within the same server chassis), the acquisition operation 926 may use DMA to acquire one or more blocks of mapped output data based on node, partition, and / or block information; or if the data resides at different nodes, the acquisition operation 926 may use Remote DMA (RDMA) over a network to acquire one or more blocks of mapped output data. In some embodiments, depending on the implementation details, the use of DMA and / or RDMA may increase throughput and / or reduce cost, latency, memory usage, and / or power consumption compared to using protocols (such as TCP / IP) over a network and / or using CPU I / O operations.

[0071] Refer again Figure 9 Once the data block is acquired using DMA / RDMA and stored in local storage, the data merging operation 930 can use a private P2P connection to transfer data from local storage (e.g., from SSD to FPGA DRAM) to merge the data and continue performing one or more shuffle operations 932. In some embodiments, the shuffle operation 932 can also be offloaded to the accelerator.

[0072] Figure 10 An embodiment of a system having a P2P connection and a DMA / RDMA engine according to the present disclosure is shown. Figure 10 The embodiments shown may include a host 1002, a storage node 1004, a communication interface 1006, a storage device 1008, an accelerator 1010, and a P2P connection 1012, which may be similar to Figure 4 The hosts, storage nodes, communication interfaces, storage devices, accelerators, and P2P connections shown are illustrated. However, Figure 10 The embodiments shown may also include a DMA / RDMA engine 1014, which enables the storage node 1004 to perform DMA transfers with devices located on the same node, for example, using an interconnect (such as PCIe). The DMA / RDMA engine may also enable the storage node 1004 to perform RDMA transfers with devices located on different nodes, for example, using a network (such as Ethernet).

[0073] Figure 11An embodiment of a method for processing data in a system having a host and storage nodes according to the present disclosure is illustrated. The method may begin at operation 1102. At operation 1104, the method may perform a shuffling operation on data stored at the storage node. Operation 1104 may include sub-operation 1106, which may include performing a shuffling write operation. Operation 1104 may also include sub-operation 1108, which may include performing a shuffling read operation. At least a portion of the shuffling operation is performed by an accelerator at the storage node in operation 1104. The method may terminate at operation 1110.

[0074] about Figure 11 The operations and / or components described in the embodiments shown herein, as well as in all other embodiments described herein, are example operations and / or components. In some embodiments, some operations and / or components may be omitted, and / or other operations and / or components may be included. Furthermore, in some embodiments, the temporal and / or spatial order of operations and / or components may be changed.

[0075] Partitioning operations

[0076] Figure 12 An embodiment of a partitioning operation for a shuffle write operation is shown. In some embodiments, Figure 12 The operations shown can be performed by the CPU. In operation 1202, the CPU can read each partition and generate a data sample for each partition. In operation 1204, the CPU can sort the samples and generate a feature splitter based on the samples. In some cases, the partitions can be resampled to generate the feature splitter. In operation 1206, the CPU can read each partition again and partition the data based on the feature splitter. In operation 1208, the CPU can continue with other operations in the shuffle write operation.

[0077] In some embodiments described above, some offloading shuffle operations or portions of offloading shuffle operations may be performed concurrently on multiple storage nodes. However, in some embodiments, partitions may share information that prevents concurrent execution.

[0078] An example implementation of a range-based partitioning algorithm can be performed as follows: (1) All or some of the dataset can be sampled to obtain K×N samples, where K can be an oversampling factor, which can be any constant value, and N can be the total number of partitions generated after partitioning. (2) The array of K×N samples can be sorted in ascending order. (3) N-1 feature splitting criteria can be obtained from the sorted K×N samples, for example, by selecting numbers at every K elements in the array. (4) All or some of the dataset can be partitioned, for example, by directly traversing the feature splitting criteria record by record (if N is small) and / or by using a binary search tree (e.g., if N is large).

[0079] In some embodiments, the partitioning algorithm can generate uniformly sized partitions for the reducer, so that the sampled data can well represent the distribution of the entire dataset. This means, for example, the more records reside in a mapping-side partition, the more samples can be generated from that partition. In some applications (e.g., big data applications), data can be distributed across multiple nodes and multiple storage devices. To offload partitions to accelerator devices while avoiding the overhead of transferring large amounts of data, embodiments of the workflow design according to this disclosure can reduce or minimize the CPU work used for coordination.

[0080] Figure 13 An example embodiment of a partitioned workflow architecture according to this disclosure is shown. Figure 13 In the embodiments shown, operations performed by the host (e.g., CPU) are generally shown on the left side of the figures, while operations performed at storage nodes (which may include accelerators and / or P2P connections between accelerators and storage devices) are shaded and generally shown on the right side of the figures, and are conceptually separated by a shading bar running through the middle of the figures.

[0081] In operation 1302, the storage node determines the number of records for one or more partitions. The storage node transmits the number of records to the host at communication 1. In operation 1304, the host determines the number of samples to be collected on a per-partition basis for one or more partitions within the partition. The host transmits the number of samples for each partition to the storage node at communication 2. In operation 1306, the storage node samples data from one or more partitions based on the number of samples determined by the host. The storage node transmits the sampled data to the host at communication 3. In operation 1308, the host sorts the sampled data and determines one or more feature partitioning criteria based on the sampled data. The host transmits the set of one or more feature partitioning criteria to the storage node at communication 4. In operation 1310, the storage node locally partitions the data into one or more partitions based on the set of feature partitioning criteria. In operation 1312, the storage node continues with other steps in the shuffle write operation.

[0082] In some embodiments, according to implementation details Figure 13 The principles illustrated can provide any number of the following advantages. First, some CPU-intensive operations (such as scanning, binary search, and / or sampling of large datasets) can be offloaded to one or more accelerators, which reduces CPU utilization. Second, data transfers between the host and storage nodes can be reduced or minimized, for example, by transferring a relatively small amount of information (such as the size of each partition, the number of samples, and / or the sample data itself) that may be smaller than the entire dataset. Additionally, in some embodiments, according to Figure 13 The overall architecture can reduce memory consumption and / or CPU utilization, for example, by efficiently offloading CPU-intensive tasks. Data parallelism within the accelerator can also be appropriately preserved using reduced or minimal CPU coordination and / or scheduling.

[0083] Refer again Figure 13 In some embodiments, a single node (e.g., a storage device) may have multiple partitions. Multiple partitions in a single storage device can be traversed in a round-robin manner (e.g., controlled by a single CPU thread). In some embodiments, the traffic load (e.g., in communications 1 through 4) can be smaller compared to scanning all records in all partitions. In some embodiments, one or more nodes may know the number of records in each partition on the node, as well as the total number of records on some or all nodes, which can be used, for example, to determine the number of samples generated proportionally for each partition on the node.

[0084] Figure 13 The embodiments shown can be described in the context of a shuffling operation using an accelerator, but the inventive principles can be applied to other applications. For example, Figure 13 The principles illustrated can be applied to sorting operations (such as external sorting), which may involve sorting large amounts of data that can be stored in one or more storage devices. In such embodiments, range partitioning rules may employ a P2P connection between the storage device and the accelerator to avoid transferring large amounts of data back and forth between the storage device and the host. In some embodiments, depending on the implementation details, this can improve parallelism, performance, and / or energy efficiency, and can reduce CPU and / or memory usage and / or network overhead.

[0085] As another example, Figure 13 The principles illustrated can be applied to range partitioning of databases. For example, one or more tables that can be the input to one or more database operations can be range-partitioned, which can be used in database engines to improve or optimize data storage and / or query performance.

[0086] In some embodiments, the principles of this disclosure can provide a general architecture for shuffling acceleration. Some embodiments may use one or more accelerators (e.g., near-memory computing devices) and / or P2P data transfer via a proprietary interconnect between the accelerator device and the storage device, and in some embodiments, DMA and / or RDMA engines may be utilized to reduce I / O and / or CPU costs. In some embodiments, depending on the implementation details, the near-memory accelerated shuffling architecture may provide any number of the following features and / or benefits.

[0087] The architecture disclosed herein can utilize enhanced storage devices and / or accelerator devices with computing power to accelerate shuffling operations, which can improve the performance of data-intensive and / or shuffling-intensive applications. Some embodiments can reduce I / O costs, memory consumption, CPU utilization, and / or network overhead, etc.

[0088] P2P communication between storage devices and accelerator devices via private interconnects can, for example, improve system scalability by not overwhelming the limited interconnect bandwidth of the host CPU.

[0089] Some embodiments can be implemented as a general shuffling acceleration architecture. Since shuffling operations may be necessary in some systems and may be a bottleneck in data processing platforms (e.g., big data), some embodiments may have broad prospects in many applications.

[0090] In some embodiments, such as FPGAs or application-specific integrated circuits (ASICs), accelerator device implementations may have lower power consumption compared to general-purpose processors, which can improve overall energy efficiency.

[0091] The embodiments disclosed above have been described in the context of various implementation details, but the principles of this disclosure are not limited to these or any other specific details. For example, some functions have been described as being implemented by specific components, but in other embodiments, functions may be distributed across different systems and components with various user interfaces in different locations. Specific embodiments have been described as having specific processes, steps, etc., but these terms also include embodiments in which specific processes, steps, etc. may be implemented by multiple processes, steps, etc., or embodiments in which multiple processes, steps, etc. may be integrated into a single process, step, etc. References to components or elements may refer to only a portion of a component or element. For example, a reference to an integrated circuit may refer to all or only a portion of an integrated circuit, and a reference to a block may refer to an entire block or one or more sub-blocks. Unless otherwise clear from the context, the use of terms such as “first” and “second” in this disclosure and claims is for the purpose of distinguishing the things they modify only and may not indicate any spatial or temporal order. In some embodiments, “based on” may mean “at least partially based on”. In some embodiments, “disabled” may mean “at least partially disabled”. A reference to a first thing may not indicate the presence of a second thing.

[0092] For convenience, various organizational aids such as chapter titles may be provided, but the subject matter arranged according to these aids and the principles of this disclosure and the embodiments described herein are not limited or restricted by these organizational aids.

[0093] The various details and embodiments described above can be combined to produce additional embodiments based on the inventive principles disclosed in this patent. Since the inventive principles disclosed in this patent can be modified in arrangement and detail without departing from the inventive concept, such changes and modifications are considered to fall within the scope of the appended claims.

Claims

1. A method for processing data in a system having a host and storage nodes, the method comprising: Perform a shuffling operation on the data stored at the storage node, wherein the shuffling operation includes: Perform a shuffle write operation, and Perform a shuffle read operation; At least a portion of the shuffling operation is performed by the accelerator at the storage node, and The at least part of the shuffle operation includes at least a part of the shuffle write operation and at least a part of the shuffle read operation.

2. The method according to claim 1, wherein, The at least part of the shuffle write operation includes a partitioning operation.

3. The method according to claim 1, wherein, The at least part of the shuffle write operation includes one or more of the following: aggregation operation, sorting operation, merging operation, serialization operation, compression operation, and overflow operation.

4. The method according to claim 1, wherein, The at least part of the shuffle read operation includes one or more of the following: an acquisition operation, a decompression operation, a deserialization operation, a merge operation, a sorting operation, and an aggregation operation.

5. The method according to any one of claims 1 to 4, wherein, At least a portion of the shuffling operation performed at the storage node includes a partitioning operation performed using a peer-to-peer network connection between the storage device and the accelerator at the storage node.

6. The method according to any one of claims 1 to 4, wherein, At least a portion of the shuffling operation performed at the storage node includes a data overflow operation performed using a peer-to-peer network connection between the storage device at the storage node and the accelerator.

7. The method according to any one of claims 1 to 4, wherein, At least a portion of the shuffling operation performed at the storage node includes an acquisition operation performed using direct memory access operations.

8. The method according to any one of claims 1 to 4, wherein, At least a portion of the shuffling operation performed at the storage node includes a data merging operation performed using a peer-to-peer network connection between the storage device at the storage node and the accelerator.

9. A storage node, comprising: Storage device; as well as accelerator; The accelerator is configured to perform at least part of the shuffling operation. The shuffling operation includes shuffling write operations and shuffling read operations, and The at least part of the shuffle operation includes at least a part of the shuffle write operation and at least a part of the shuffle read operation.

10. The storage node according to claim 9, wherein, The storage node also includes a peer-to-peer network connection between the storage device and the accelerator; and The storage device and accelerator are configured to perform at least a portion of the shuffling operation by transmitting data via a peer-to-peer network connection.

11. The storage node according to claim 10, wherein, Accelerators are integrated with storage devices; and The storage node includes a server.

12. The storage node according to claim 10, wherein, The storage device is a first storage device, the accelerator is a first accelerator, the peer-to-peer network connection is a first peer-to-peer network connection, and the storage node further includes: Second storage device; The second accelerator; and A second peer-to-peer network connection between the second storage device and the second accelerator; The second storage device and the second accelerator are configured to perform at least a portion of the shuffling operation by transmitting data via a second peer-to-peer network connection.

13. The storage node according to claim 12, wherein, The first and second storage devices are configured to perform at least a portion of the shuffling operation by transferring data via a direct memory access operation.

14. The storage node according to any one of claims 9 to 13, wherein, The storage node is configured to perform at least a portion of the shuffling operation by transferring data to an additional storage node via a remote direct memory access operation.

15. A method for partitioning data, the method comprising: Data from one or more partitions is sampled at the device based on the number of samples. The sampled data is transmitted from the device to the host computer; At the host, one or more feature partitioning criteria are determined based on the sampled data; The one or more feature classification criteria are transmitted from the host to the device; as well as The device partitions the data of the one or more partitions based on the one or more feature partitioning criteria. The shuffling operation includes a shuffling write operation and a shuffling read operation. At least a portion of the shuffling operation is performed by an accelerator at the device. The at least a portion of the shuffling operation includes at least a portion of the shuffling write operation and at least a portion of the shuffling read operation, and the at least a portion of the shuffling write operation includes the method.

16. The method of claim 15, further comprising: The number of records in the one or more partitions is determined at the device; The number of records in the one or more partitions is transmitted from the device to the host. Determine the number of samples for the one or more partitions at the host; as well as The number of samples is transmitted from the host to the device.

17. The method according to claim 15 or claim 16, wherein, The sampling is performed by an accelerator at the device.

18. A system for accelerating mixed washing, comprising: Storage nodes, including accelerators; and The host is configured to perform a shuffling operation on the data stored at the storage node. The shuffling operation includes shuffling write operation and shuffling read operation; The storage node is configured to use an accelerator to perform at least a portion of the shuffling operation, and The at least part of the shuffle operation includes at least a part of the shuffle write operation and at least a part of the shuffle read operation.

Images