Optical AllReduce Routing Using Idle GPU Relay Paths

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Distributed machine learning (DML) training is bottlenecked by low network bandwidth in traditional cloud services, particularly in large language models (LLMs), where network overhead accounts for up to 60% of training iteration time, and existing routing procedures for AllReduce transfers are inefficient.

Innovation Solution

A collaborative routing procedure that leverages unused bandwidth resources from single or standalone GPU servers to provision indirect optical paths with a two-hop communication, maximizing bandwidth allocation and minimizing communication hops for AllReduce transfers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If traditional cloud services with standard routing are used for DML training, then network coverage is comprehensive, but network bandwidth is low and training speed is slow

Engineering Contradiction:

Improvetraining speedVSAvoidnetwork bandwidth efficiency

Core Design Contradiction:

SpeedVSLoss of energy

Solution Approach 1:

The patent introduces optical circuit switching as an intermediary layer between GPUs to establish dedicated high-bandwidth optical paths. This mediator enables direct optical connections that bypass traditional electrical interconnect limitations, achieving up to 100x bandwidth improvement and directly accelerating DML training speed while improving network bandwidth efficiency

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces traditional electrical routing mechanisms with optical circuit switching. By substituting electrical signals with optical signals and using optical switches instead of electrical routers, the system achieves dramatically higher bandwidth and lower latency, directly resolving the training speed bottleneck caused by limited network bandwidth

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Quantity of substance

If direct optical paths are established between GPUs for AllReduce transfers, then bandwidth allocation is maximized, but the number of communication hops increases and bandwidth tax increases

Engineering Contradiction:

Improvebandwidth allocationVSAvoidcommunication hops

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The patent applies different routing strategies to different communication patterns. For AllReduce operations, it establishes direct optical paths with dedicated bandwidth allocation. For other traffic types, it uses indirect paths through available GPUs. This local optimization ensures that critical DML training traffic receives maximum bandwidth while minimizing communication hops

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent implements dynamic path selection that adapts to real-time network conditions. The system continuously monitors bandwidth availability and communication requirements, dynamically establishing direct optical paths when needed and using indirect paths when resources are constrained. This dynamic approach optimizes both bandwidth allocation and communication efficiency

Inventive Principle:

Principle #15Dynamics

3Quantity of substance

If unused bandwidth resources from single GPU servers are leveraged for indirect optical paths, then bandwidth allocation improves, but routing complexity increases

Engineering Contradiction:

Improvebandwidth allocationVSAvoidrouting procedure complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent enables idle single-GPU servers to automatically contribute their unused bandwidth resources to the optical network. These servers self-configure as relay nodes, providing indirect optical paths without requiring manual intervention. The system automatically discovers and utilizes these self-service resources, improving bandwidth allocation while keeping routing management simple

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent creates a universal routing framework that handles both direct and indirect optical paths through a unified control mechanism. The same optical circuit switching infrastructure serves multiple functions: direct high-speed paths for AllReduce operations and indirect relay paths utilizing idle GPU resources. This multi-functionality simplifies the overall routing procedure by using a single versatile system rather than separate specialized pathways

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20260044467A1Efficient routing procedure for accelerating distributed machine learning models in optical circuit switching based cloud

Publication Date: 2026.02.12 NEC LABORATORIES AMERICA INC

AI summary

Disclosed are techniques that provide efficient routing strategies for AllReduce transfers, which are the the dominant traffic in machine learning-centric datacenters, resulting in faster parameter synchronization in distributed machine learning and improving the average training time by over 9%. As compared with the prior art, our efficient route of AllReduce traffic advantageously maximizes bandwidth allocation while minimizing bandwidth tax, accelerates training speed of distributed machine learning models or large language models in optical circuit switching-based clouds, and more efficiently provisions indirect optical paths, by leveraging the unused ports or bandwidth resources from GPU servers that run single or standalone computing jobs.