19 results about "Collective operation" patented technology

Examples described herein relate to a network interface device that includes: a host interface; a direct memory access (DMA) circuitry; a network interface to receive, in at least one packet, time data associated with at least one of multiple layers, wherein the multiple layers provide inputs to a collective operation associated with a large language model (LLM); and circuitry. The circuitry is to based, at least in part, on the time data associated with the multiple layers, identify a first operation of a first layer of the multiple layers as a late completing process relative to times to completion of multiple first operations of other layers and based on the first operation being identified as a late completing process, perform a remedial action to adjust at least one configuration of a first device to execute a second operation of the first layer.

The disclosed device can perform a collective operation on received datasets, and split the result into chunks in accordance with a chunking scheme. The device can also forward the chunks in accordance with a routing scheme that can direct chunks to appropriate nodes of a collective network. Various other methods, systems, and computer-readable media are also disclosed.

Systems, methods, and machine-readable media may facilitate programmable data trimming. Metadata associated with a collective operation may be determined by an application of a server. The metadata may specify a job identifier corresponding to a unit of work to be completed in conjunction with the collective operation, a collective type of the collective operation, and / or an ordering mode for packets corresponding to the collective operation. The metadata associated with the collective operation may be sent by the application to a network interface card (NIC). The NIC may be caused by the application to transmit a data packet with the metadata embedded in a cookie of the data packet to a switch of a network fabric to cause the switch to use a selected network path and / or selected load-balancing for the collective operation based on one or more of the job identifier, the collective type, and / or the ordering mode.

Automatic run-time skew-aware optimization of rooted collectives includes determining skews amongst compute nodes of a distributed computing system, as an application executes on the compute nodes, and determining implementations for collective operations of the application based at least in part on the skews. Skews may be determined based on timestamps of operations of the application program. Time stamps of one or more of the compute nodes may be estimated or inferred from timestamps of other compute nodes. Timestamps may be aggregated to determine global skews. Collective implementations may be determined for a sequence of collective operations based on skew impacts amongst the sequence of collective operations. Subsequent collective operations may be predicted based on current collective operations and a history of persistent collective operations, and implementations may be determined for the predicted collective operations prior to receipt of calls for the predicted collective operations.

A method for performing a message passing interface (MPI) collective operation in a network, wherein the network comprises a plurality of interconnected nodes, the method comprising: receiving, at a node of the plurality of interconnected nodes, MPI collective operation information identifying the MPI collective operation to be performed, and a graph of the network; determining a number of algorithmic steps of the MPI collective operation based on the MPI collective operation and the graph of the network; determining an initialisation process for the algorithmic steps; determining a finalisation process for the algorithmic steps; determining, for each of the algorithmic steps: a subset of nodes of the plurality of interconnected nodes for the node to communicate with; one or more portions of data for the node to send to and receive from the nodes within the subset of nodes; and initialising the MPI collective operation based on the determined subset, initialisation process and finalisation process, and the one or more portions of data.

Examples described herein relate to a network interface device that includes: a host interface; a direct memory access (DMA) circuitry; a network interface to receive, in at least one packet, time data associated with at least one of multiple layers, wherein the multiple layers provide inputs to a collective operation associated with a large language model (LLM); and circuitry. The circuitry is to based, at least in part, on the time data associated with the multiple layers, identify a first operation of a first layer of the multiple layers as a late completing process relative to times to completion of multiple first operations of other layers and based on the first operation being identified as a late completing process, perform a remedial action to adjust at least one configuration of a first device to execute a second operation of the first layer.

Systems, methods, apparatuses, and computer program products for zero-copy symmetric multicast communication buffers for offload operations. A method may include receiving an instruction for performing a collective operation across a plurality of processing elements. The method may also include determining an offset of the first virtual address from a first base address associated with the first set of contiguous virtual addresses. The method may further include translating the first virtual address to a corresponding multicast virtual address based on the offset. Further, the method may include causing the collective operation to be performed based at least on the multicast virtual address.

Systems, methods, and devices for performing computational operations and managing network congestion are provided. An example describes a device comprising a processing unit that collects multiple messages and performs an operation as part of a collective operation on the data contained within the multiple messages. It then generates an output message containing the result of the operation performed on the data contained within the multiple messages. The processing unit can further embed a congestion notification in the output message if at least one of the multiple messages also contains a corresponding congestion notification.

A system receives a first request to perform a collective operation. The system stores a mapping of a first virtual address to a descriptor for a physical location of an allocated memory region. The system performs the collective operation, by writing data to a first segment of the memory region and accessing data from other segments of the memory region. The system receives a second request to perform an update operation, the second request indicating the first virtual address, one or more portions of a memory region segment to be updated, and corresponding data units to write to the portions. The system updates, based on the mapping, only the indicated portions by writing the corresponding data units. The system performs a subsequent iteration of the collective operation, based on the mapping, by bypassing writing any data to the memory region and only accessing data units from the memory region.

The disclosed device includes a collectives engine that can offload collectives communications of multiple nodes and perform collective operations. The collectives engine can manage a direct mapping scheme of local memories of the nodes for access by the collectives engine. Various other methods, systems, and computer-readable media are also disclosed.

In some examples, a processor receives a first request to allocate a memory region for a collective operation by process entities in a plurality of computer nodes. In response to the first request, the processor creates a virtual address for the memory region and allocates the memory region in a network-attached memory coupled to the plurality of computer nodes over a network. The processor correlates the virtual address to an address of the memory region in mapping information. The processor identifies the memory region in the network-attached memory by obtaining the address of the memory region from the mapping information using the virtual address in a second request. In response to the second request, the processor performs the collective operation.

An apparatus facilitating efficient key refresh in a node is provided. During operation, the apparatus can determine a collective operation initiated by the node. The node can include a processor and can be in a distributed system comprising a plurality of nodes. The collective operation can be performed by a subset of the plurality of nodes in conjunction with each other. The apparatus can generate a new key based on a previous key maintained at the apparatus. Here, a respective key can be used for encrypting an inter-node packet in the distributed system. The apparatus can maintain the new and previous keys for the duration of the collective operation. Either of the new and previous keys can be used for decrypting messages received at the apparatus from other nodes of the distributed system. Upon determining a threshold point of the collective operation, the apparatus can discard the previous key.

A processing system dynamically and selectively quantizes or compresses variable-length input data for a collective operation to fit within a predetermined limit, executes the collective operation on the compressed data, and converts the data back to variable length results by dequantizing or decompressing the results of the collective operation.

Techniques for collective operation and compute operation pipelining on integrated circuit devices configured to perform operations associated with machine learning models are described herein. A first node or integrated circuit device can generate a first input subtensor and a second input subtensor based on the first input tensor. Several collective operations and compute operations can be performed on the first input subtensor and second input subtensor in combination with input subtensors from other integrated circuit devices to generate output subtensors. The first node can generate an output tensor based on the output subtensors.

The disclosed device includes a collective engine that can select a communication cost model from multiple communication cost models for a collective operation and configure a topology of a collective network for performing the collective operation using the selected communication cost model. Various other methods, systems, and computer-readable media are also disclosed.

The disclosure relates to a plurality of network routers for performing collective operations and an accelerator system including the network routers. A network router includes: a receiver configured to receive a first input packet from a first other network router in a first direction, receive a second input packet from a second other network router in a second direction, and output one of the first input packet or the second input packet as a collective packet; a network controller configured to receive the collective packet from the receiver and output the collective packet through a first path or a second path based on a packet type of the collective packet; a buffer circuit configured to store, in a distinguishable manner, the collective packet transferred from the network controller through the second path based on a packet type of the collective packet; and a reduction operation circuit configured to receive the collective data packet stored in the buffer circuit, and perform a reduction operation using the received collective data packet.

The examples described herein relate to a network interface device comprising: a host interface; a direct memory access (DMA) circuit arrangement; a network interface for receiving, in at least one packet, time data associated with at least one of several layers, the several layers providing inputs to a collective operation associated with a large language model (LLLM); and a circuit arrangement.The circuit arrangement is designed to identify, at least partially based on the timing data associated with the multiple layers, a first operation of a first layer of the multiple layers as a late-completed process relative to the times until the completion of multiple first operations of other layers, and, based on the first operation being identified as a late-completed process, to perform a remedial action to adapt at least one configuration of a first device to perform a second operation of the first layer.

The present disclosure relates to non-reliable, out-of-order data transfers over a memory interconnect architecture. An inter-process communication method divides a total payload of a first memory command among a plurality of packets and encodes a last transmitted packet of the plurality of packets to include a metadata code in the total payload. The memory command can be transmitted between processes that perform a collective operation. Payloads of the packets are written to memory in a sequential address order such that the metadata code is written to a particular address and a second process generates an acknowledgement to the first memory command to the first process on a condition that the metadata code is read back from the particular address.

An AI-accelerating processor system may include an acyclic subset of hardware processing nodes. The acyclic subset includes a plurality of end nodes that are disconnected from other end nodes in the acyclic subset. The acyclic subset of hardware processing nodes is configured to perform, according to schedules, computations that are part of a collective operation. A first hardware processing node in the subset has a first scheduling pattern and a second hardware processing node in the subset has a second scheduling pattern that is different from the first scheduling pattern to account for the subset being acyclic. The acyclic subset of hardware processing nodes is also configured to transmit computation outputs to neighboring hardware processing nodes among the acyclic subset through the bi-directional links to generate a result that is part of the collective operation. The result is contributed by each of the hardware processing nodes in the acyclic subset.