Smart NIC Serverless Placement for Low-Latency Multi-Tenant Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing serverless architectures face challenges in executing functions close to data sources due to their ephemeral nature, leading to inefficiencies in latency, scalability, and security when using traditional host processors.
Innovation Solution
Utilizing smart network interface cards (NICs) with integrated processors and commodity storage to create containers for executing serverless functions, allowing for multi-tenant optimization by executing functions near data ingress points, leveraging PCIe bus access and agent policies for efficient resource management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If serverless functions are executed on traditional host processors, then infrastructure complexity is reduced, but latency increases and scalability is limited
Solution Approach 1:
The patent extracts the function execution capability from the traditional host processor and relocates it to the network interface card. By taking out the computational workload from the host and placing it in the NIC, the system achieves lower latency execution closer to data sources while the host maintains simplified infrastructure management responsibilities.
Solution Approach 2:
The patent introduces a new dimension for function execution by utilizing the network interface card's integrated processors, which are traditionally unused or underutilized resources. This dimensional shift from host-only execution to NIC-based execution enables closer proximity to data and network operations, reducing latency without significantly increasing overall infrastructure complexity.
2Speed
If serverless functions are executed close to data sources, then latency is reduced, but security risks increase
Solution Approach 1:
The patent introduces an intermediary security layer through the containerization mechanism that wraps function execution on the NIC. The container acts as a mediator between the function code and the NIC hardware, providing isolation and security controls that prevent direct access to critical system resources while still enabling low-latency execution close to data sources.
Solution Approach 2:
The system implements self-service security mechanisms where the NIC-based function execution environment automatically isolates and manages its own security boundaries through containerization. The execution environment serves its own security needs by creating isolated execution contexts that prevent harmful code from affecting the broader system while maintaining the performance benefits of proximate execution.
3Productivity
If network interface cards execute functions, then scalability is enhanced, but resource management complexity increases
Solution Approach 1:
The patent applies universality by enabling the network interface card to perform multiple functions: traditional network data processing plus executing serverless functions. This multi-functionality allows the same hardware resource to serve dual purposes, enhancing scalability by utilizing previously underutilized NIC processing capacity without requiring separate dedicated function execution hardware.
Solution Approach 2:
The system implements feedback mechanisms for resource management through the containerization framework that monitors and manages NIC resource allocation. The feedback loop enables dynamic adjustment of resource allocation based on workload demands, allowing the system to scale functions across multiple NICs while automatically managing resource distribution to optimize performance and prevent resource exhaustion.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Multi-tenant optimized serverless placement using network interface card and commodity storage may be provided. A first request to execute a first function may be received. Next, it may be determined to execute the first function at a first network interface card. The first network interface card may include a plurality of processors. Then, a container may be created at the first network interface card. The container may have at least one processor of the plurality of processors. The first function may be executed at the container.