Serverless function startup acceleration method and system based on CXL memory hierarchical storage

Abstract

This invention discloses a method and system for accelerating serverless function startup based on CXL memory tiered storage. By introducing a CXL memory tiered storage architecture, this invention significantly shortens the cold start latency of serverless functions and constructs a two-stage loading mechanism to overcome the performance bottleneck of traditional full image loading. The system utilizes the high bandwidth and low latency characteristics of CXL to quickly prefetch only the critical working set essential for function execution into physical contiguous memory blocks on the local computing node, ensuring that core instructions and data are available instantly within a millisecond window, while non-core data is lazily mapped in remote extended memory. This strategy not only eliminates most of the time overhead of network transmission and disk reads, achieving instantaneous response of computational logic, but also effectively reduces the peak usage of local physical memory, achieving efficient utilization of storage resources while ensuring the integrity of the virtual memory space.

Description

Technical Field

[0001] This invention relates to the field of function computing, specifically to a serverless function startup acceleration method and system based on CXL memory tiered storage. Background Technology

[0002] While existing serverless computing architectures greatly simplify application development and deployment processes with their on-demand allocation and elastic scaling capabilities, they often face significant cold start latency challenges when dealing with sudden traffic spikes or low-frequency call scenarios. When a function instance is created for the first time or triggered again after a long period of inactivity, the cloud platform must go through a series of sequential operations, including allocating computing resources, downloading code packages, starting the container environment, and initializing runtime dependencies, resulting in a significant increase in end-to-end response time.

[0003] To mitigate this issue, existing technologies typically employ resource reservation or instance preheating strategies. This involves using predictive algorithms to pre-start and maintain a certain number of warm instances in a standby state, allowing for direct reuse of the existing environment when user requests arrive. However, this strategy inherently sacrifices resource utilization for performance. Maintaining idle instances for extended periods not only occupies valuable memory and computing resources, significantly increasing service costs, but also makes it difficult to guarantee the accuracy of the predictive algorithms. Furthermore, it cannot completely eliminate the impact of cold starts when facing unpredictable load fluctuations.

[0004] To further reduce startup overhead, snapshot-based checkpoint recovery technology has become another mainstream solution. This technology saves the memory state and register context after function initialization as an image file, and restores the state directly from the persistent storage medium to memory during subsequent startup, thereby skipping the cumbersome initialization process.

[0005] While the methods described above avoid repetitive computational initialization steps, in practical applications, function image files are typically large. Loading the full image data from remote object storage or network disk into local memory is limited by network bandwidth and disk I / O performance. Especially in large-scale concurrent startup scenarios, massive data migration becomes a new performance bottleneck. Furthermore, existing technologies mostly employ a full sequential loading mode, meaning execution can only begin after the image data is completely read into memory. This makes it difficult to overcome the physical limitations of snapshot-based methods in reducing data transfer latency. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a Serverless function startup acceleration method and system based on CXL memory tiered storage.

[0007] The Serverless function startup acceleration method based on CXL memory tiered storage in this invention includes the following steps:

[0008] S1. Critical Working Set Identification: Based on the hardware access bits of the processor page table entries, a background scanning process is started to periodically traverse the function process page table, track and count the memory page access frequency, identify the critical working set with the highest access frequency on the startup and core execution paths, rearrange the critical working set into a contiguous memory block in the physical address space and persistently store it in an extended memory pool that supports the CXL protocol.

[0009] S2. Critical working set prefetching: When a function call request is received that requires a cold start, the high bandwidth and low latency characteristics of the CXL interconnect channel are used to directly access and transfer the critical working set in the extended memory pool to the local memory of the compute node.

[0010] S3, Lazy State Remote Mapping: For non-core lazy state pages in the function image, manipulate the page table structure to point their physical address to a remote CXL memory address and set it to read-only permission, build a zero-overhead remote mapping, so that the function process can start execution immediately after obtaining a complete view of the virtual memory space;

[0011] S4. Background Silent Migration: During function execution, a low-priority background merging thread is started. A bandwidth-aware strategy is used to utilize idle memory bandwidth to silently migrate lazy pages in remote CXL memory to local memory, transforming remote memory access into local memory access.

[0012] S5. Copy-on-write processing: When a function executes and writes to a lazy page, a write protection exception is triggered. A new physical page frame is allocated in local memory, the target page data is copied from the remote CXL memory to the local physical page, the page table entry is updated to point to the new local physical page and modified to read and write permissions, and instruction execution is resumed.

[0013] S6. Model self-update: After the function finishes execution, local memory resources are released, and the scope of the critical working set is fine-tuned and updated based on the access pattern data collected in this run. Frequently accessed lazy pages are included in the critical working set, forming a self-optimization closed loop.

[0014] The Serverless Function Startup Acceleration System based on CXL Memory Tiered Storage in this invention includes:

[0015] Critical Working Set Identification Module: Based on the hardware access bits of the processor page table entries, a background scanning process is started to periodically traverse the function process page table, track and count the memory page access frequency, identify the critical working set with the highest access frequency on the startup and core execution paths, rearrange the critical working set into a contiguous memory block in the physical address space and persistently store it in an extended memory pool that supports the CXL protocol.

[0016] Critical working set prefetching module: When a function call request is received that requires a cold start, the high bandwidth and low latency characteristics of the CXL interconnect channel are used to directly access and transfer the critical working set in the extended memory pool to the local memory of the compute node.

[0017] Lazy state remote mapping module: For non-core lazy state pages in the function image, manipulate the page table structure to point their physical addresses to remote CXL memory addresses and set them to read-only permissions, build a zero-overhead remote mapping, so that the function process can start execution immediately after obtaining a complete view of the virtual memory space;

[0018] Background silent migration module: During function execution, a low-priority background merging thread is started. A bandwidth-aware strategy is used to utilize idle memory bandwidth to silently migrate lazy pages in remote CXL memory to local memory, transforming remote memory access into local memory access.

[0019] Copy-on-write processing module: When a function executes and writes to a lazy page, a write protection exception is triggered. A new physical page frame is allocated in local memory, the target page data is copied from the remote CXL memory to the local physical page, the page table entry is updated to point to the new local physical page and modified to read and write permissions, and instruction execution is resumed.

[0020] Model self-update module: After the function finishes execution, it releases local memory resources, fine-tunes and updates the critical working set range based on the access pattern data collected in this run, and includes frequently accessed lazy pages into the critical working set, forming a self-optimization closed loop.

[0021] This invention significantly reduces the cold start latency of Serverless functions by introducing a CXL-based memory tiered storage architecture and constructs a two-stage loading mechanism to overcome the performance bottleneck of traditional full image loading. Leveraging CXL's high bandwidth and low latency, the system rapidly prefetches only the critical working set essential for function execution into physical contiguous memory blocks on the local compute node, ensuring that core instructions and data are available instantly within millisecond windows, while non-core data is lazily mapped in remote extended memory. This strategy not only eliminates most of the time overhead from network transmission and disk reads, achieving instantaneous response of computational logic, but also effectively reduces the peak usage of local physical memory, achieving efficient utilization of storage resources while ensuring the integrity of the virtual memory space.

[0022] Furthermore, this invention possesses extremely low system invasiveness in runtime optimization capabilities. By employing asynchronous profiling technology based on hardware page table access bits, it can accurately identify and dynamically update memory usage characteristics without interfering with business logic or requiring modification of user code, thus constructing a closed-loop self-optimization model. During function execution, the system combines zero-overhead read-only mapping and copy-on-write mechanisms to ensure data consistency, and utilizes a bandwidth-aware strategy to initiate a background silent migration thread, gradually converting remote cold data into local hot data without competing for foreground computing resources. This design not only provides transparent services to upper-layer applications but also continuously learns access pattern differences to ensure continuous improvement in the startup performance and execution efficiency of subsequent function calls. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the prior art and embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] This application proposes a serverless function startup acceleration method and system based on CXL memory tiered storage to solve the problem of high cold start latency in existing serverless computing. The system mainly consists of three core parts: an asynchronous parser, a two-stage loader, and a runtime memory manager.

[0026] Among them: the asynchronous parser is used to accurately identify the core memory pages required for function execution in a non-intrusive environment; the two-stage loader uses CXL technology to load memory states in stages during cold start; and the runtime memory manager is responsible for maintaining memory mapping and handling write operations and background data migration during function execution.

[0027] Furthermore, the two-stage loader, as the core execution unit of this invention, breaks the traditional full-load mode and adopts a strategy of local prefetching of critical working sets combined with remote mapping of lazy sets to ensure the immediate availability of core path instructions and data, while utilizing the high bandwidth characteristics of CXL to carry non-core data.

[0028] The specific steps of the embodiments of this application are as follows: Figure 1 As shown, the details are as follows:

[0029] Step 1: Activate the non-intrusive asynchronous profiling mechanism to identify the critical working set based on hardware characteristics.

[0030] This embodiment first introduces a non-intrusive asynchronous profiling mechanism based on hardware characteristics to accurately identify critical working sets during function warm-up or first run phases. Instead of traditional code instrumentation monitoring, this process utilizes the hardware access bit characteristics in the processor's page table entries for low-overhead analysis. The system starts a background scanning process that runs independently of the function's main logic, periodically traversing the function's page table structure.

[0031] Specifically, the background scanning process reads and resets the access bits in the page table entries. By continuously tracking and statistically analyzing which memory pages' access bits are frequently set during function initialization and core business logic execution, the system can construct an accurate memory access heatmap. Based on the statistical results of multiple observations, the system defines the set of memory pages that are accessed most frequently and are essential on the startup and core execution paths as the critical working set.

[0032] Furthermore, after identifying the critical working set, the system performs a memory rearrangement operation at the physical storage level. This operation reorganizes these logically potentially scattered critical working sets into contiguous memory blocks in the physical address space and persistently stores them in an extended memory pool that supports the CXL protocol, thereby providing a solid physical foundation for subsequent high-speed direct memory access or batch prefetch operations.

[0033] Step 2: The two-stage loader performs the first stage of loading, completing the local prefetching of the critical working set through the CXL channel.

[0034] When a Serverless function call request is received and the system determines that a cold start is required, the two-phase loader immediately intervenes and executes the first phase of loading, namely the local prefetching of the critical working set. In this phase, the system abandons the traditional practice of loading the complete function image from slow media such as object storage or network disks.

[0035] The two-stage loader leverages the high bandwidth and low latency of the CXL interconnect channel to directly initiate direct memory access transfers. The critical working set, pre-organized in the extended memory pool in step 1, is prefetched into the compute node's local dynamic random access memory at extremely high speed.

[0036] Since the critical working set typically occupies only a small portion of the function's total memory footprint, and the data is physically laid out in a continuous manner, this transfer process is extremely fast. This ensures that the core code segments, stack space, and hot global variables required for function execution are ready within a millisecond-level time window, thereby eliminating most of the input / output waiting time in the traditional cold start process.

[0037] Step 3: The two-stage loader performs the second-stage loading, building a zero-overhead remote mapping for lazy state pages.

[0038] After completing the local loading of the critical working set, the two-phase loader then enters the second phase, constructing zero-overhead remote mappings for the remaining massive amount of non-core pages in the function image. For this portion of data, known as the lazy state, the system does not perform actual data transfer operations, but instead directly manipulates the operating system's page table structure.

[0039] Specifically, the loader traverses the virtual address range of the remaining pages, modifies their corresponding page table entries, and points their physical addresses to the remote CXL memory addresses. More importantly, the loader sets the access permissions of these pages to read-only. This operation only involves updating the page table metadata and does not incur any large-scale data copy overhead; therefore, it is called zero-overhead mapping.

[0040] After completing this step, from a logical perspective, the function process has a complete view of the virtual memory space, while from a physical perspective, non-core data still resides in a remote extended memory device. At this point, the function's main thread can immediately begin executing instructions, and the system has successfully reduced the blocking latency of a cold start to a level comparable to the time required to load a small amount of critical working data.

[0041] Step 4: The runtime memory manager starts a background merge thread to perform silent data migration to optimize long-term performance.

[0042] As the main thread of the function begins executing business logic, the runtime memory manager starts a low-priority background merging thread to optimize memory access performance for long-running tasks. This thread remains active throughout the function's execution and is responsible for gradually and silently migrating lazy pages from remote CXL memory to local dynamic random access memory.

[0043] This process employs an intelligent bandwidth-aware strategy, utilizing only the idle memory bandwidth of compute nodes for data transfer. The aim is to gradually transform remote memory access into local memory access, thereby eliminating the minor latency caused by cross-CXL access. Because the merging thread is strictly limited to low priority, it does not compete for the processor computing resources and main memory bandwidth resources necessary for foreground function execution, thus achieving gradual optimization of the business logic execution environment without the user's awareness.

[0044] Step 5: Handle lazy page write operations and trigger the copy-on-write exception mechanism to ensure data consistency.

[0045] During function execution, if a program instruction attempts to write to a lazy page that is still residing in CXL memory, the system will trigger the copy-on-write exception mechanism to ensure data integrity.

[0046] Because all page table entries pointing to remote memory are marked as read-only during the second-stage mapping construction described in step 3, any write operation will be intercepted by the processor's memory management unit, triggering a page fault or write protection exception. The runtime memory manager will capture this exception signal and immediately suspend the execution of the current instruction. Subsequently, the manager will request a new physical page frame in local dynamic random access memory and copy the original data of the target page from the remote extended memory into this new local physical page.

[0047] After data replication is complete, the manager updates the page table entry, pointing the mapping of the virtual page to the new local physical page, and simultaneously changes the page's access permissions to read and write. Finally, the system resumes the execution of the suspended instructions, thus ensuring program data consistency while only incurring replication costs when actual data modification is required, achieving an effective balance between startup latency and runtime overhead.

[0048] Step 6: Resource reclamation and self-updating of the critical working set model after function execution.

[0049] Once all business logic of a function has been executed, the system enters the resource reclamation and state update phase. The runtime memory manager releases the local memory resources occupied by the function and writes modified dirty data back to backend storage or discards it according to the persistence strategy.

[0050] Meanwhile, the asynchronous parser uses the latest access pattern data collected during this run to fine-tune and update the scope of the critical working set. The system determines whether any pages that were originally classified as lazy were frequently accessed or written during this run. If so, they are marked and included in the critical working set during the next image reassembly, thus forming a self-optimizing closed loop to ensure continuous optimization of the performance of subsequent function cold starts.

[0051] Based on the above-mentioned isomorphic concept, embodiments of this application also provide a Serverless function startup acceleration system based on CXL memory tiered storage, including:

[0052] Critical Working Set Identification Module: Based on the hardware access bits of the processor page table entries, a background scanning process is started to periodically traverse the function process page table, track and count the memory page access frequency, identify the critical working set with the highest access frequency on the startup and core execution paths, rearrange the critical working set into a contiguous memory block in the physical address space and persistently store it in an extended memory pool that supports the CXL protocol.

[0053] Critical working set prefetching module: When a function call request is received that requires a cold start, the high bandwidth and low latency characteristics of the CXL interconnect channel are used to directly access and transfer the critical working set in the extended memory pool to the local memory of the compute node.

[0054] Lazy state remote mapping module: For non-core lazy state pages in the function image, manipulate the page table structure to point their physical addresses to remote CXL memory addresses and set them to read-only permissions, build a zero-overhead remote mapping, so that the function process can start execution immediately after obtaining a complete view of the virtual memory space;

[0055] Background silent migration module: During function execution, a low-priority background merging thread is started. A bandwidth-aware strategy is used to utilize idle memory bandwidth to silently migrate lazy pages in remote CXL memory to local memory, transforming remote memory access into local memory access.

[0056] Copy-on-write processing module: When a function executes and writes to a lazy page, a write protection exception is triggered. A new physical page frame is allocated in local memory, the target page data is copied from the remote CXL memory to the local physical page, the page table entry is updated to point to the new local physical page and modified to read and write permissions, and instruction execution is resumed.

[0057] Model self-update module: After the function finishes execution, it releases local memory resources, fine-tunes and updates the critical working set range based on the access pattern data collected in this run, and includes frequently accessed lazy pages into the critical working set, forming a self-optimization closed loop.

[0058] Based on the above-described methodological isomorphism, this application also provides a function startup acceleration device, including: a memory for storing computer programs; and a processor for executing the steps of the described function startup acceleration method.

[0059] Based on the above-mentioned isomorphic concept, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the function startup acceleration method.

[0060] Furthermore, it should be noted that in this specification, the terms "include," "contain," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, the inclusion of a defined element by a statement does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0061] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for accelerating the launching of Serverless functions based on CXL memory hierarchical storage, characterized in that The method includes the following steps: S1. Critical Working Set Identification: Based on the hardware access bits of the processor page table entries, a background scanning process is started to periodically traverse the function process page table, track and count the memory page access frequency, identify the critical working set with the highest access frequency on the startup and core execution paths, rearrange the critical working set into a contiguous memory block in the physical address space and persistently store it in an extended memory pool that supports the CXL protocol. S2. Critical working set prefetching: When a function call request is received that requires a cold start, the high bandwidth and low latency characteristics of the CXL interconnect channel are used to directly access and transfer the critical working set in the extended memory pool to the local memory of the compute node. S3, Lazy State Remote Mapping: For non-core lazy state pages in the function image, manipulate the page table structure to point their physical address to a remote CXL memory address and set it to read-only permission, build a zero-overhead remote mapping, so that the function process can start execution immediately after obtaining a complete view of the virtual memory space; S4. Background Silent Migration: During function execution, a low-priority background merging thread is started. A bandwidth-aware strategy is used to utilize idle memory bandwidth to silently migrate lazy pages in remote CXL memory to local memory, transforming remote memory access into local memory access. S5. Copy-on-write processing: When a function executes and writes to a lazy page, a write protection exception is triggered. A new physical page frame is allocated in local memory, the target page data is copied from the remote CXL memory to the local physical page, the page table entry is updated to point to the new local physical page and modified to read and write permissions, and instruction execution is resumed. S6. Model self-update: After the function finishes execution, local memory resources are released, and the scope of the critical working set is fine-tuned and updated based on the access pattern data collected in this run. Frequently accessed lazy pages are included in the critical working set, forming a self-optimization closed loop.

2. The method of claim 1, wherein, The hardware access bits mentioned in step S1 include access bits and dirty page bits in the processor page table entries. The background scanning process periodically reads and resets the access bits to count the memory page access frequency.

3. The method according to claim 1 or 2, characterized in that, The critical working set mentioned in step S1 includes the set of memory pages corresponding to code segments, stack space, and hot global variables that are accessed during the function initialization phase and the execution of core business logic.

4. The method of claim 1, wherein, The direct memory access transfer described in step S2 bypasses traditional object storage or network disks and initiates memory-to-memory data transfer directly through the CXL.mem protocol.

5. The method of claim 1, wherein, The zero-overhead remote mapping described in step S3 only involves updating page table metadata and does not generate actual data copying, allowing the main thread of the function to start execution within a millisecond-level time window.

6. The method according to claim 1 or 5, characterized in that, The write protection exception mentioned in step S5 is triggered by the processor memory management unit intercepting the write operation of the read-only page. After the runtime memory manager captures the exception, it performs the copy-on-write process.

7. The method of claim 1, wherein, The fine-tuning update in step S6 includes: determining whether there are pages classified as lazy that are frequently accessed or written during the current run; if so, marking them and including them in the critical working set during the next mirror reassembly.

8. A Serverless function launch acceleration system based on CXL memory tiering storage, characterized in that, include: Critical Working Set Identification Module: Based on the hardware access bits of the processor page table entries, a background scanning process is started to periodically traverse the function process page table, track and count the memory page access frequency, identify the critical working set with the highest access frequency on the startup and core execution paths, rearrange the critical working set into a contiguous memory block in the physical address space and persistently store it in an extended memory pool that supports the CXL protocol. Critical working set prefetching module: When a function call request is received that requires a cold start, the high bandwidth and low latency characteristics of the CXL interconnect channel are used to directly access and transfer the critical working set in the extended memory pool to the local memory of the compute node. Lazy state remote mapping module: For non-core lazy state pages in the function image, manipulate the page table structure to point their physical addresses to remote CXL memory addresses and set them to read-only permissions, build a zero-overhead remote mapping, so that the function process can start execution immediately after obtaining a complete view of the virtual memory space; Background silent migration module: During function execution, a low-priority background merging thread is started. A bandwidth-aware strategy is used to utilize idle memory bandwidth to silently migrate lazy pages in remote CXL memory to local memory, transforming remote memory access into local memory access. Copy-on-write processing module: When a function executes and writes to a lazy page, a write protection exception is triggered. A new physical page frame is allocated in local memory, the target page data is copied from the remote CXL memory to the local physical page, the page table entry is updated to point to the new local physical page and modified to read and write permissions, and instruction execution is resumed. Model self-update module: After the function finishes execution, it releases local memory resources, fine-tunes and updates the critical working set range based on the access pattern data collected in this run, and includes frequently accessed lazy pages into the critical working set, forming a self-optimization closed loop.

9. A function launch acceleration device, comprising: include: Memory, used to store computer programs; A processor for implementing the steps of the function startup acceleration method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the function startup acceleration method as described in any one of claims 1 to 7.

Images