Network Interface Device In-Network Compute for ML Reproducibility

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In machine learning (ML) training using parameter server (PS) architectures, the non-associative and order-dependent nature of floating-point (FP) operations leads to reproducibility issues due to unpredictable and out-of-order packet arrival at network interface devices, which can hinder accurate aggregation and computation of model updates.

Innovation Solution

A network interface device functions as both a transport and security protocol endpoint, reordering FP data for reproducible operations, decrypting and encrypting packets as needed, and performing arithmetic operations like accumulation and multiplication to ensure consistent results across parallel computations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If packets are transmitted across the network for distributed ML training, then parallel computation and data aggregation are enabled, but packet arrival order becomes unpredictable leading to reproducibility issues in floating-point operations

Engineering Contradiction:

Improveparallel computation capabilityVSAvoidreproducibility of FP operations

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The network interface device performs preliminary reordering of incoming packets based on sequence numbers before passing them to the processing unit. This preliminary action ensures that floating-point operations receive data in the correct order, guaranteeing reproducibility while maintaining parallel computation capabilities across the distributed system

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The network interface device acts as an intermediary between the network and the ML training system. It mediates the data flow by reordering packets and managing sequence numbers, thereby isolating the upper-layer applications from the unpredictable network arrival order and ensuring reproducible FP operations

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If encryption protocols are implemented for secure communications, then data security is enhanced, but processing overhead and computational complexity increase

Engineering Contradiction:

Improvedata securityVSAvoidprocessing overhead

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent combines multiple functions into the network interface device: packet reception, encryption/decryption, sequence number management, and reordering. By merging these functions into a single device, the system reduces overall processing overhead and avoids redundant operations that would occur if these functions were distributed across multiple components

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The network interface device performs self-service by autonomously managing the encryption and decryption of packets without requiring intervention from higher-layer protocols. This self-service capability reduces processing overhead by handling security operations at the network interface level rather than requiring application-level processing

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20230379309A1In-network compute operations utilizing encrypted communications

Publication Date: 2023.11.23 INTEL CORP

AI summary

Examples described herein relate to an interface and circuitry coupled to the interface. The circuitry can provide an endpoint for a PSP Security Protocol (PSP) connection to a first network interface device, provide an endpoint for a second PSP connection with a second network interface device, provide a transport layer endpoint for the packets received from the first network interface device, and provide a second transport layer endpoint for the packets received from the second network interface device.