Virtual Server Identity in Multitenant Serverless Execution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud-based service provider networks face challenges in efficiently managing resources for applications with fluctuating demand, as dedicating full-time hardware resources is wasteful but provisioning too few resources leads to overwhelm during peaks, necessitating a more flexible execution system to reduce costs.

Innovation Solution

Implementing a multitenant serverless execution service (MTSES) with a request labeling endpoint that dynamically assigns worker nodes and provides a server or endpoint ID to virtual servers, allowing for on-demand resource provisioning without static hardware allocation, decoupling virtual servers from physical machines and enabling scalable, cost-effective resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If full-time hardware resources are dedicated to applications with fluctuating demand, then the applications can handle peak demand without overwhelm, but hardware resources remain idle during low demand periods causing waste

Engineering Contradiction:

Improveability to handle peak demandVSAvoididle hardware resources

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The system dynamically provisions hardware resources based on real-time demand conditions. Worker nodes are allocated to virtual servers only when needed, and can be reassigned or deallocated when demand decreases, allowing the system to adapt resource allocation continuously rather than maintaining static dedicated resources

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

Hardware resources are made universal by allowing worker nodes to serve multiple virtual servers across different tenants. A single worker node can be dynamically assigned to different virtual servers based on demand, enabling one resource to fulfill multiple functions and serve multiple applications sequentially

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Loss of energy

If hardware resources are reduced to eliminate waste, then cost is reduced, but applications may be overwhelmed during peak demand periods

Engineering Contradiction:

Improvehardware resource utilizationVSAvoidability to handle peak demand

Core Design Contradiction:

Loss of energyVSReliability

Solution Approach 1:

The system maintains a pool of standby worker nodes that are pre-configured and ready to be allocated immediately when demand increases. These nodes are prepared in advance with necessary software environments and can be rapidly assigned to virtual servers without significant provisioning delay, ensuring peak demand is met without over-provisioning for average load

Inventive Principle:

Principle #10Preliminary action

3Ease of operation

If static hardware allocation is used, then resource provisioning is simple and stable, but flexibility and scalability are reduced for handling fluctuating demand

Engineering Contradiction:

Improveresource provisioning simplicityVSAvoidflexibility for fluctuating demand

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system implements automated resource allocation where the orchestration layer automatically provisions, assigns, and deallocates worker nodes to virtual servers based on demand signals. This self-service mechanism eliminates the need for manual resource provisioning while maintaining operational simplicity, allowing the system to scale and adapt automatically without complex manual intervention

Inventive Principle:

Principle #25Self-service

4Loss of energy

If dynamic resource allocation is implemented, then resource utilization efficiency is improved, but system complexity increases

Engineering Contradiction:

Improvehardware resource utilization efficiencyVSAvoidresource management system complexity

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

An orchestration layer is introduced as an intermediary between hardware resources and virtual servers. This intermediary manages the complexity of dynamic resource allocation, handling worker node provisioning, assignment, monitoring, and deallocation automatically. By centralizing control logic in this intermediary layer, the complexity is encapsulated and managed systematically rather than being distributed across individual components

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11201915B1Providing virtual server identity to nodes in a multitenant serverless execution service

Publication Date: 2021.12.14 AMAZON TECH INC

AI summary

Computer systems and methods are disclosed to implement a virtual server using a multitenant serverless execution service (MTSES) and a request labeling endpoint outside the MTSES to identify the virtual server to worker nodes in the service. The MTSES is configured to host multiple virtual servers, and forward requests for the different virtual servers to dynamically selected worker nodes. However, in embodiments, the MTSES is not configured to determine the virtual server targeted by the requests. Accordingly, to allow the MTSES to support virtual servers that require knowledge of their identity, for example to authorize incoming requests, a request labeling endpoint is provided outside the MTSES to forward requests to the MTSES and automatically label individual requests with an endpoint identifier associated with their target virtual servers. In this manner, nodes in the MTSES can dynamically determine the virtual server that they are to assume for each request.