Serverless Function Cache Policy for Cold Start and Resource Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing serverless computing systems face challenges in balancing the frequency of cold start invocations with cache resource efficiency, leading to unnecessary storage costs and resource wastage due to indiscriminate instance caching.
Innovation Solution
A centralized cache management system using a primary controller to determine and adjust cache policies across computing nodes based on global information, distinguishing between hotspot and non-hotspot functions, and dynamically adjusting cache durations to optimize resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If nodes cache as many instances as possible to minimize cold starts, then cold start invocation frequency is reduced, but cache redundancy increases and node resources are wasted
Solution Approach 1:
The patent changes the parameter of cache policy from a static, uniform approach to a dynamic, differentiated approach based on function hotness. The primary controller adjusts cache policies for different computing nodes based on global information about function invocation patterns, transforming the cache management from a one-size-fits-all model to a customized model that adapts to actual usage patterns.
Solution Approach 2:
The patent segments the cache management system into a centralized primary controller and distributed sub-controllers on computing nodes. It also segments the cache policy application by function type (hotspot vs. non-hotspot functions), allowing different caching strategies for different function categories. This segmentation enables precise control over what is cached where, reducing unnecessary cache redundancy.
2Reliability
If indiscriminate instance caching is performed, then cold start invocations are reduced, but storage costs and resource wastage increase
Solution Approach 1:
The primary controller collects global information about function invocation patterns and uses this feedback to dynamically adjust cache policies. By monitoring which functions are frequently invoked (hotspot functions) and which are not, the system continuously optimizes cache allocation, ensuring that cache resources are concentrated on functions that actually benefit from caching while avoiding waste on rarely used functions.
Solution Approach 2:
The patent applies different cache policies to different computing nodes and different function types based on local characteristics. Hotspot functions receive aggressive caching policies, while non-hotspot functions receive minimal or no caching. This local quality approach ensures that cache resources are allocated where they provide the most value, balancing service availability with resource efficiency.
Data Source
AI summary
Embodiments of this application provide a function cache method and apparatus. The method includes: A primary controller sends first indication information to a first sub-controller in a plurality of sub-controllers, where the first indication information is used to determine a cache policy for an instance on a first computing node in a cluster; the first sub-controller performs instance caching on the instance on the first computing node based on the first indication information; the primary controller sends second indication information to a second sub-controller in the plurality of sub-controllers, where the second indication information is used to determine a cache policy for an instance on a second computing node in the cluster; and the second sub-controller performs instance caching on the instance on the second computing node based on the second indication information.