Network Switch AI Inference Architecture for Low-Latency Services
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning infrastructure in cloud-based networks faces challenges in providing transparent, powerful, and power-efficient AI capabilities across various communication paths, including edge computing and central office data centers, with high latency requirements and inefficient data movement.
Innovation Solution
A switch architecture with integrated AI service logic and inference resources that breaks out AI services locally, utilizing neural networks and compute resources to reduce data movement and meet low latency demands, supported by management and load balancing logic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If AI services are processed at central office data centers, then computational power is available, but data movement distance increases and latency worsens
Solution Approach 1:
The patent segments AI service processing across multiple locations including edge devices, base stations, and central office data centers. This allows different AI services to be processed at different locations based on latency requirements and computational needs, resolving the contradiction between having powerful computational resources and maintaining low latency.
Solution Approach 2:
The patent introduces a hierarchical architecture that adds a spatial dimension to AI service processing. Instead of processing all AI services at a single central location, the system creates multiple processing levels from edge to core network, enabling closer processing for latency-sensitive services while maintaining access to centralized computational power for less time-critical services.
2Adaptability or versatility
If more AI services are deployed across the network, then service coverage improves, but data movement requirements increase
Solution Approach 1:
The patent extracts AI service processing capabilities from the central office and deploys them at edge locations including base stations and user equipment. This extraction reduces the amount of data that needs to be moved to central offices for processing, as more AI services can be executed locally at the edge where the data originates.
Solution Approach 2:
The patent implements local AI service processing at different network locations based on specific requirements. Edge devices perform AI services locally when latency is critical or data privacy is important, while central offices handle services requiring substantial computational power. This localized approach reduces unnecessary data movement while maintaining comprehensive service coverage.
3Loss of time
If AI services are processed locally at edge devices, then latency is reduced, but computational resources at edge locations are limited
Solution Approach 1:
The patent creates a multi-functional AI service architecture where edge devices, base stations, and central offices can all participate in AI service processing. The system can route services to the location with appropriate computational resources based on the specific service requirements, allowing edge devices to handle latency-critical services while central offices provide computational power when needed.
Solution Approach 2:
The patent introduces base stations and network elements as intermediary components between edge devices and central offices. These intermediaries can offload AI services from computationally limited edge devices to more powerful central office resources when necessary, while still enabling low-latency processing when edge computational resources are sufficient.
Data Source
AI summary
Examples include techniques for artificial intelligence (AI) capabilities at a network switch. These examples include receiving a request to register a neural network for loading to an inference resource located at the network switch and loading the neural network based on information included in the request to support an AI service to be provided by users requesting the AI service.