Network Interface Device Reordering for Encrypted ML Training

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In machine learning (ML) training using parameter server (PS) architectures, the non-associative and order-dependent nature of floating-point (FP) arithmetic operations leads to reproducibility issues due to unpredictable and out-of-order packet arrival at network interface devices, which can affect the accuracy and reliability of model training.

Innovation Solution

A network interface device is designed to serve as both a transport and security protocol endpoint, capable of reordering FP data for reproducible operations, decrypting and encrypting packets, and performing arithmetic operations like accumulation and multiplication, ensuring that FP data is processed in a predictable order, regardless of arrival order, using cryptographic protocols like TLS, DTLS, and PSP.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If encrypted packet transmission is used for secure communication, then data security is improved, but packet arrival order becomes unpredictable due to network conditions

Engineering Contradiction:

Improvedata securityVSAvoidoperation reproducibility

Core Design Contradiction:

ReliabilityVSManufacturing precision

Solution Approach 1:

The network interface device serves as an intermediary between the network and the compute device. It receives encrypted packets, decrypts them, reorders them based on sequence numbers to ensure correct operation order, and then processes them. This intermediary function resolves the contradiction by handling both security (through decryption) and reproducibility (through reordering) at the network interface layer.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system performs preliminary actions by embedding sequence numbers in packet headers before transmission and preparing reorder buffers at the network interface device. These preliminary structures enable the device to later reorder packets correctly without affecting the security of the encrypted payload, thus ensuring reproducible operations despite out-of-order arrival.

Inventive Principle:

Principle #10Preliminary action

2Productivity

If packets are transmitted in parallel across the network, then transmission speed is improved, but packets arrive out of order affecting FP operation reproducibility

Engineering Contradiction:

Improvetransmission speedVSAvoidFP operation reproducibility

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The system segments the data transmission by dividing floating-point data into multiple packets with individual sequence numbers. Each packet can be transmitted independently in parallel, maintaining high throughput. The network interface device uses these sequence numbers to reassemble packets in the correct order before processing, thus preserving FP operation reproducibility while enabling parallel transmission.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The invention changes the parameter of packet organization by introducing sequence numbers as a new parameter in the packet header. This parameter enables the receiving device to sort and reorder packets correctly, transforming the unordered parallel arrival into ordered sequential processing without reducing transmission parallelism.

Inventive Principle:

Principle #35Parameter changes

3Device complexity

If network interface devices perform only transport functions, then device simplicity is maintained, but security protocol endpoint functionality is lost

Engineering Contradiction:

Improvenetwork interface device functionalityVSAvoidsecurity protocol endpoint capability

Core Design Contradiction:

Device complexityVSReliability

Solution Approach 1:

The network interface device is designed with multi-functionality, serving both as a transport layer endpoint and a security protocol endpoint. It performs multiple functions including packet reception, encryption/decryption, reordering based on sequence numbers, and floating-point arithmetic operations. This universal design resolves the contradiction by consolidating multiple functions in one device, enhancing security capability without requiring separate dedicated security devices.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20230379154A1In-network compute operations utilizing encrypted communications

Publication Date: 2023.11.23 INTEL CORP

AI summary

Examples described herein relate to an interface and circuitry coupled to the interface. The circuitry can provide an endpoint for a Datagram Transport Layer Security (DTLS) connection with a first network interface device, provide an endpoint for a second DTLS connection with a second network interface device, provide a transport layer endpoint for the packets received from the first network interface device, and provide a second transport layer endpoint for the packets received from the second network interface device.