Programmable Data Plane ML Inferencing for Low-Latency Packets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deploying machine learning algorithms in integrated circuit devices for network functionalities such as congestion control and anomaly detection results in high latency due to data transportation and high resource consumption, and varying feature requirements complicate throughput mapping.
Innovation Solution
Implementing a system architecture with feature extraction and inferencing circuitry on an integrated circuit device, utilizing programmable logic to enable flexible feature extraction and per-packet or per-flow inferencing, reducing latency and resource consumption through offload processing and co-processing systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If feature extraction and model inferencing are implemented in a CPU, then flexibility for feature extraction is improved, but latency increases due to data transportation to and from the CPU
Solution Approach 1:
The patent introduces a co-processing system that acts as an intermediary between the CPU and data plane. This co-processor handles feature extraction and ML inferencing locally, eliminating the need to transport data to and from the CPU while maintaining the flexibility of programmable feature extraction through software-configurable processing logic.
Solution Approach 2:
The system segments the processing architecture by separating feature extraction and ML inferencing functions from the main CPU. These functions are implemented in dedicated hardware blocks within the data plane, allowing independent optimization of each processing stage while maintaining overall system flexibility through programmable interfaces.
2Loss of time
If feature extraction is implemented in a fixed hardware block, then latency is reduced, but computing resources and power consumption increase
Solution Approach 1:
The patent implements dynamically reconfigurable hardware blocks that can be programmed to perform different feature extraction functions based on the specific workload requirements. This allows the system to use hardware acceleration only when needed, reducing power consumption compared to always-on fixed hardware while maintaining low latency through hardware-based processing when active.
Solution Approach 2:
The system changes the operational parameters of the hardware blocks by allowing dynamic reconfiguration of feature extraction pipelines. This enables the same hardware infrastructure to adapt to different algorithmic requirements, optimizing the balance between processing speed and resource utilization for varying workloads.
3Adaptability or versatility
If features are extracted for a large number of flows, then network problem solving capability is improved, but device complexity increases
Solution Approach 1:
The patent implements universal feature extraction blocks that can handle multiple types of network flows and protocols through a single programmable architecture. Instead of requiring separate dedicated hardware for each flow type, the system uses reconfigurable processing units that can be programmed to extract features from various protocols (TCP, UDP, ICMP, etc.), reducing overall device complexity while maintaining comprehensive network analysis capability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems or methods of the present disclosure may provide an integrated circuit system including data plane circuitry that includes a packet processing pipeline and extraction circuitry, wherein the extraction circuitry is configurable to extract one or more features of a packet via the packet processing pipeline. The integrated circuit system also includes inferencing circuitry configurable to perform machine learning inferencing based on the one or more features.