Container resource occupancy overhead optimization method, system, medium, and computer device

Abstract

The present invention proposes a method, system, medium, and computer device for optimizing container resource usage overhead. The method includes: recording the files accessed during the execution of the function process corresponding to each container, generating a file working set for each container; retaining only the files in the file working set for a single container and deleting other files, and identifying duplicate files between different file working sets for multiple containers to perform deduplication processing; generating a snapshot file for each container and pausing the operation of the container, identifying duplicate memory pages in each snapshot file to perform deduplication processing; and restoring the container from the snapshot file when a new container needs to be started. This method improves the degree and efficiency of resource sharing between containers on the same server, achieving the effect of reducing container resource overhead and accelerating container startup speed.

Description

Technical Field

[0001] The present invention belongs to the field of serverless computing technology, and specifically relates to a container resource occupancy overhead optimization method, system, medium, and computer equipment for serverless computing scenarios. Background Art

[0002] In serverless computing, developers don't need to worry about underlying server management and resource configuration; they only need to focus on writing business logic, leaving the cloud service provider to dynamically allocate and manage computing resources. In serverless computing, containerization technology is widely used to improve application portability, scalability, and resource utilization. Containerization packages an application and its dependencies into a portable container, which can be deployed and run in different environments. This approach allows developers to manage and deploy applications more flexibly while providing a more consistent operating environment.

[0003] However, the current containerized deployment approach for serverless computing scenarios has some shortcomings:

[0004] (1) Excessive storage overhead: The image file size of each container is large, and there is a large amount of duplicate storage between the file systems of multiple containers, resulting in excessive storage overhead.

[0005] (2) Serious waste of memory resources: In serverless computing scenarios, function containers are usually kept alive for a period of time after processing a function call request, and are used to process function call requests in the future. Only when no requests arrive within a period of time will the container be recycled. Therefore, multiple containers can exist on a computing node at the same time. However, the memory resource management of each container is relatively independent, resulting in a large amount of memory resource waste, especially when there are a large number of files and memory data that can be shared between containers.

[0006] (3) High cold start latency: Since file resources and memory resources are not fully shared between containers, there are redundant initialization operations when starting a new container, resulting in high container cold start latency.

[0007] Therefore, the current containerized deployment method has certain limitations in terms of resource utilization efficiency and performance, and a more efficient and economical resource management method is needed to optimize containerized deployment. Most related technical solutions handle storage overhead or memory overhead independently, without fully utilizing the relationship between the two. This leads to high computational complexity when performing file deduplication or memory deduplication, and incomplete file and memory sharing between containers after deduplication. The above problems are not comprehensively considered and solved. At present, there is an urgent need to propose a method for comprehensively optimizing the storage overhead, memory overhead, and startup delay of serverless computing containers to address the shortcomings of existing technical solutions and improve resource utilization and system performance. Summary of the Invention

[0008] In response to the shortcomings of the existing technology, the present invention proposes a method, system, medium, and computer equipment for optimizing container resource usage overhead in multi-faceted serverless computing scenarios, which improves the degree and efficiency of resource sharing between containers on the same server, thereby reducing container resource overhead and accelerating container startup speed.

[0009] To achieve the above objectives, the present invention provides a method for optimizing container resource usage overhead, comprising:

[0010] Record the files accessed during the execution of the function process corresponding to each container, and generate the file working set of each container;

[0011] For a single container, only the files in the file working set are retained and other files are deleted. For multiple containers, duplicate files between different file working sets are identified for deduplication.

[0012] Generate a snapshot file for each container and suspend the operation of the container, identify duplicate memory pages in each snapshot file for deduplication;

[0013] When you need to start a new container, restore the container from the snapshot file.

[0014] In one embodiment, for a file working set of a single container, redundant files outside the file working set are deleted to retain only the files within the file working set.

[0015] In one embodiment, duplicate files between different file working sets are deduplicated, and only one copy of the duplicate files is retained. The remaining duplicate files are replaced with hard links pointing to the copy, and which files are replaced with hard links are recorded.

[0016] In one embodiment, identifying duplicate files between different file working sets for multiple containers includes:

[0017] Mark all files with the same path name in different file working sets as suspected duplicate files;

[0018] For all suspected duplicate files, sample file data blocks of the same size at the same offset to obtain the sampled data blocks corresponding to each suspected duplicate file;

[0019] A hash value is calculated for each sampled data block. If the hash values ​​of sampled data blocks of different suspected duplicate files are the same, the suspected duplicate file is confirmed as a duplicate file.

[0020] In one embodiment, deduplicating memory pages in the snapshot file includes:

[0021] For duplicate memory pages, only one copy is retained in one snapshot file, duplicate copies in other snapshot files are deleted, and the location of the retained copy is recorded.

[0022] In one embodiment, identifying duplicate memory pages in each snapshot file includes:

[0023] In the case where the memory pages in the snapshot file include anonymous memory pages,

[0024] For anonymous memory pages, the hash value of each anonymous memory page is calculated, and the data of the anonymous memory pages with the same hash value are compared byte by byte. If the contents of two pages are exactly the same, they are identified as duplicate memory pages.

[0025] In one embodiment, when the memory pages in the snapshot file include non-anonymous memory pages,

[0026] For a non-anonymous memory page, query whether a first file corresponding to the non-anonymous memory page is replaced with a hard link;

[0027] If it has been replaced by a hard link, it is considered that the non-anonymous memory page corresponding to the first file is the same as the memory page corresponding to the second file pointed to by the hard link and has been merged, and the non-anonymous memory page is identified as a duplicate memory page;

[0028] If it is not replaced by a hard link, the hash value of each non-anonymous memory page is calculated, and the data of the non-anonymous memory pages with the same hash value are compared byte by byte. If the contents of the two pages are exactly the same, they are identified as duplicate memory pages.

[0029] In one embodiment, each snapshot file is stored in memory, and each snapshot includes mapping information from the file to the memory;

[0030] It is queried based on the file-to-memory mapping information whether the first file corresponding to the non-anonymous memory page is replaced with a hard link.

[0031] In one embodiment, restoring the container from the snapshot file includes:

[0032] For snapshot files that have not been modified during the deduplication phase, the container is restored directly from the snapshot file;

[0033] For the snapshot file modified in the deduplication phase, the copy is read according to the location of the retained copy, and the initial snapshot file is restored and then the container is recovered from the initial snapshot file.

[0034] In one embodiment, after receiving a function call request, the survival monitoring module in each container generates a function process to process the function call request and starts monitoring file access information of the function process;

[0035] After the function process completes processing the function call request, all file access information of the function process throughout its life cycle is recorded to construct a working set of the file.

[0036] Another aspect of the present invention provides a container resource usage overhead optimization system, which uses the above-mentioned container resource usage overhead optimization method and at least includes:

[0037] A survival monitoring module records the files accessed during the execution of the function process corresponding to each container and generates a file working set for each container.

[0038] The file deduplication module is used to retain only the files in the file working set of a single container and delete other files. It also identifies duplicate files between different file working sets of multiple containers for deduplication processing.

[0039] A memory deduplication module, which generates a snapshot file for each container and pauses the container's operation, identifying duplicate memory pages in each snapshot file for deduplication processing;

[0040] Call the module to restore the container from the snapshot file when a new container needs to be started.

[0041] In addition, the present invention also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the above-mentioned container resource occupancy overhead optimization method.

[0042] Furthermore, the present invention provides a computer-readable storage medium storing at least one program code, which is loaded and executed by a processor to implement the container resource occupancy overhead optimization method as described above.

[0043] From the above scheme, it can be seen that the advantages of the present invention are:

[0044] The container resource occupancy overhead optimization method disclosed by the present invention addresses the resource occupancy redundancy problem among multiple containers in the current serverless computing scenario. The method obtains file access information during the execution of a function container, and based on this information, streamlines the file system of a single container, and deduplicates the memory and file system of all containers on the same computing node, thereby improving the degree and efficiency of resource sharing between containers on the same server, achieving the effect of reducing container resource overhead and accelerating container startup speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The overall process diagram of the container resource usage overhead optimization method is shown;

[0046] Figure 2 The figure shows the proportion of file working sets of different function containers in the root file system.

[0047] Figure 3 Shows the impact of deduplication on memory consumption and startup time;

[0048] Figure 4 A framework diagram corresponding to a container resource usage overhead optimization system is shown.

[0049] in:

[0050] 30: Container resource usage overhead optimization system

[0051] 31: Survival monitoring module

[0052] 32: Deduplication module

[0053] 321: File deduplication module

[0054] 322:Memory deduplication module

[0055] 33:Calling module

[0056] 34A: First container

[0057] 34B: Second container

[0058] S1-S4: steps. DETAILED DESCRIPTION

[0059] In order to make the above features and effects of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

[0060] See Figure 1-Figure 3 As shown in Figure 1 The overall process diagram of the container resource usage overhead optimization method is shown. Figure 2 It shows the proportion of file working sets of different function containers in the root file system. Figure 3 Shows the impact of deduplication on memory consumption and startup time.

[0061] See also Figure 1 As shown in , the container resource usage overhead optimization method specifically includes the following steps:

[0062] Step S1: Record the files accessed when the function process corresponding to each container is executed, and generate a file working set for each container.

[0063] In a serverless computing scenario, it becomes difficult to capture the file access information of a function container. This is because the execution time of a function process in a container is usually very short, so it is impossible to obtain the container's open file list at any time, as in traditional long-running application containers. To this end, in one embodiment, a survival monitoring module Sniffer component is abstracted from each container as the parent process of the function process. After receiving a function call request, the survival monitoring module generates a function process to process the function call request and begins to monitor the file access information of the function process. After the function process processes the function call request, all file access information of the function process throughout its life cycle is recorded to construct a file working set. In this way, the problem of difficulty in capturing file access information due to the short running time of the function process in a serverless computing scenario is solved, and accurate capture of the file access information of the function process throughout its life cycle is achieved.

[0064] Regarding the optimization of storage overhead, memory overhead, and startup latency of containers in serverless computing scenarios, most related technologies handle storage overhead or memory overhead independently without leveraging the relationship between the two, resulting in high computational complexity when performing file deduplication or memory deduplication and incomplete file and memory sharing between containers after deduplication. In response to this, in this embodiment, the container's file access information, namely the file working set, is used to efficiently deduplicate the container's file data and memory data, including the file deduplication process in step S2 and the memory deduplication process in step S3, to reduce storage overhead and memory overhead.

[0065] Step S2: For a single container, only the files in the file working set are retained and other files are deleted. For multiple containers, duplicate files between different file working sets are identified for deduplication.

[0066] Redundant container storage overhead comes from two sources: a. For a single container, the current container image packaging method results in a large number of files within the image that are unnecessary for the container to run. These files are not accessed by the function applications within the container, occupying a large amount of storage space. b. For multiple containers, the file working sets of different containers may contain a large number of identical files, i.e., duplicate files, which exponentially consumes server storage space. Related technologies do not distinguish whether files within the container image are accessed. Instead, they calculate the hash value of each file within the image, identify duplicate files by comparing the hash values, and then perform deduplication. This introduces a large amount of unnecessary calculations, resulting in inefficient deduplication and failing to meet the serverless computing requirements for fast container startup and release.

[0067] In this regard, in this embodiment, based on the file access information of the container, only the files in the file working set of a single container are retained and other files are deleted; for multiple containers, duplicate files between different file working sets are identified for deduplication processing.

[0068] In the specific implementation, for a single container's file working set, redundant files outside the file working set are deleted to retain only the files within the file working set. For file working sets of multiple containers, duplicate files between different file working sets are deduplicated. For duplicate files, only one copy is retained, and the remaining duplicate files are replaced with hard links pointing to the copy. The files that are replaced with hard links are recorded.

[0069] Regarding how to identify duplicate files within a file working set, in one embodiment, all files with the same path name within different file working sets of multiple containers are marked as suspected duplicate files. For all suspected duplicate files, file data blocks of the same size are sampled at the same offset to obtain the sampled data blocks corresponding to each suspected duplicate file. A hash value is calculated for each sampled data block. If the hash values ​​of the sampled data blocks of different suspected duplicate files are the same, the suspected duplicate file is confirmed as a duplicate file. This duplicate file identification method reduces the number of files that need to be compared and the computational complexity of comparing two files, eliminating the need to calculate and compare the hash values ​​of the entire file one by one, thereby improving the efficiency of deduplication in the container file system.

[0070] In this embodiment, while streamlining a single container file system is achieved, the efficiency of deduplication of multiple container file system files is improved, and the storage overhead of the container file system on the computing node is reduced.

[0071] Step S3: Generate a snapshot file for each container and suspend the operation of the container, identify duplicate memory pages in each snapshot file for deduplication processing.

[0072] In addition to the container's image files, duplicate data also exists in the container's memory data. The memory of a process is divided into anonymous memory and non-anonymous memory. Anonymous memory is usually the stack segment of the process, which is dynamically allocated using the memory allocation function and does not correspond to a specific file; non-anonymous memory is a memory area created by memory-mapped files. Related memory deduplication solutions, such as Kernel Same Page Merging (KSM) used by the Linux kernel, only target anonymous memory. KSM first calculates the hash value of the page. If the hash values ​​are the same, the page data is compared byte by byte. If the contents of the two pages are exactly the same, then they are considered to be the same page. If there are identical pages in non-anonymous memory, KSM cannot identify them. If KSM is simply used to scan all memory pages, including non-anonymous memory pages, the computational overhead is very high and cannot meet the requirements of serverless computing for fast startup and release of containers.

[0073] To address this issue, this embodiment takes a snapshot of the container process, saves the container's memory data as a snapshot file, and stores each snapshot file in memory. The snapshot file contains the container's memory page data and file-to-memory mapping information. Different duplication detection methods are used for anonymous and non-anonymous container memory pages in the snapshot file, and duplicate memory pages are identified in each snapshot file for deduplication.

[0074] Specifically, for the anonymous memory pages contained in the snapshot file, the method of calculating hash values ​​and comparing byte by byte is still used to determine whether they are duplicate pages. That is, for the anonymous memory pages, the hash value of each anonymous memory page is calculated, and the data of the anonymous memory pages with the same hash value are compared byte by byte. If the contents of the two pages are exactly the same, they are identified as duplicate memory pages.

[0075] For the non-anonymous memory pages contained in the snapshot file, the above-mentioned container file working set information is used to efficiently determine whether duplication occurs for the non-anonymous memory pages. Specifically, the first file corresponding to the non-anonymous memory page is queried based on the file-to-memory mapping information to determine whether it has been replaced with a hard link; if it has been replaced with a hard link, it is considered that the non-anonymous memory page corresponding to the first file is the same as the memory page corresponding to the second file pointed to by the hard link and has been merged, and the non-anonymous memory page is identified as a duplicate memory page; if it has not been replaced with a hard link, the hash value of each non-anonymous memory page is calculated, and the data of the non-anonymous memory pages with the same hash value are compared byte by byte. If the contents of the two pages are exactly the same, they are identified as duplicate memory pages. In this embodiment, the container snapshot file is used to extend the existing memory merging technology (KSM) to non-anonymous memory, and the file mapping information of the non-anonymous memory is used to improve the efficiency of memory deduplication for multiple containers. After the deduplication operation, the memory pages in all container snapshots on the node are no longer duplicated, reducing the total container memory overhead on the computing node.

[0076] Furthermore, in this embodiment, for duplicate memory pages identified by the above method, only one unique copy is retained in one snapshot file, duplicate copies in other snapshot files are deleted, and the location of the retained copy is recorded. This method speeds up the identification of duplicate memory pages and improves the efficiency of deduplication in container memory.

[0077] Step S4: When a new container needs to be started, the container is restored from the snapshot file.

[0078] In this embodiment, after the file system deduplication process of step S2 and the memory deduplication process of step S3, it is necessary to start the container and restore the container from the snapshot file. When restoring the container, if the snapshot file has not been modified during the deduplication stage, the container is directly restored from the snapshot file; if the snapshot file has been modified during the deduplication stage, the location of the deleted memory page is queried based on the retained copy location, the deleted memory page is read to restore the initial snapshot file, and then the operation of the container is restored from the initial snapshot file. Since the container snapshot file already contains memory data, the container no longer needs to initialize all memory data when it is started from the snapshot file; and the files corresponding to the non-anonymous memory pages have been merged, so multiple containers can share memory pages through the page cache mechanism of the operating system, and a page cache hit will occur when a newly started container reads the shared memory page. The above two points accelerate the startup of the container.

[0079] In summary, the container resource usage overhead optimization method disclosed in the present invention optimizes the storage overhead, memory overhead and startup delay of containers in serverless computing scenarios. The present invention uses the container's file access information to achieve efficient deduplication of the container's file data and memory data, reducing storage overhead and memory overhead, and allowing the deduplicated containers to share memory pages through the operating system's page cache, thereby accelerating subsequent container startup. Figure 2 This demonstrates the effectiveness of the present invention in reducing file system storage overhead for a single container. We selected five different function containers—cnn_serving, Ir_serving, inception, inception-pip, and detect—and captured their file working sets. We found that the file working set only accounted for 30% to 40% of the total file system size. Figure 3 It reflects the effect of the present invention on reducing memory overhead and accelerating container startup for multiple containers. The following applies the inception container to the same function, and uses 10 different ways to build the image, including using the official basic image to build, using source code to build, using the package manager of different operating systems to install dependencies, local packaging, etc. (this is possible in actual development), to obtain 10 container file systems with a large number of duplicate files but not shared with each other. Start the function container one by one from these 10 container file systems, and send function call requests one by one to trigger the deduplication operation. After a round of deduplication, clear the operating system page cache, resend the function call requests one by one, and compare the memory consumption and container startup time of the containers before and after deduplication. It can be seen that compared with the container without deduplication, after the first container processes the request, the memory consumption of subsequent containers when processing the request is reduced, and the startup time is reduced.

[0080] In addition, corresponding to the embodiment of the above-mentioned container resource occupancy overhead optimization method, the present invention also provides an embodiment of a container resource occupancy overhead optimization system. Figure 4 , an embodiment of the present invention provides a container resource occupancy overhead optimization system 30, which runs on a serverless computing node. The system includes two roles: a survival monitoring module (sniffer) 31, a deduplication module 32, and a calling module 33. The deduplication module 32 specifically includes a file deduplication module 321 and a memory deduplication module 322. Among them, the survival monitoring module 31 runs in the container, and is used to accept function call requests, generate function processes, monitor the file access status of function processes, form the file working set of the container and report it to the deduplication module. The deduplication module 32 runs outside the container, and is used to accept the container file working set reported by the sniffer, and perform file deduplication and memory deduplication operations on all containers on the computing node. The file deduplication module 321 is used to retain only the files in the file working set of each container and delete other files, and identify duplicate files in the file working set for deduplication processing. The memory deduplication module 322 is used to generate a snapshot file for each container and suspend the operation of the container, and identify duplicate memory pages in each snapshot file for deduplication processing. The calling module 33 is used to restore the container from the snapshot file when a new container needs to be started.

[0081] The operation process of the system is described with a specific embodiment. Specifically, on the computing node, the deduplication module 32 is started; at least two function containers containing the survival monitoring module are started, the first container 34A and the second container 34B. Without loss of generality and for the convenience of description, it is assumed that there are two containers, as shown in the attached figure. Figure 4, respectively, the first container 34A and the second container 34B, there are a large number of duplicate files in the two containers; the calling module 33 initiates function call requests to the first container 34A and the second container 34B respectively. After receiving the function call, the survival monitoring module 31 in the two containers generates a function process to process the function call request and starts to monitor the file access information of the function process; after the function process processes the function call request, the survival monitoring module 31 reports the file access information of the function process (i.e., the container file working set) to the deduplication module 32; after receiving the file working set of the container, the deduplication module 32 generates a snapshot file for the corresponding container and ends the operation of the corresponding container. The snapshot file is stored in the memory and contains the memory data of the container and the mapping information of the file to the memory; the deduplication module 32 records the file working sets of the two containers in the deduplication file list, deletes the files outside the file working set of the corresponding container, and marks the suspected duplicate files according to the deduplication file list; the deduplication module 32 filters out the indeed duplicate files from the suspected duplicate files, performs deduplication operations on the indeed duplicate files, and marks the merged files in the deduplication file list; without loss of generality, it is assumed that among the duplicate files, the files of the first container 34A are retained, and the files of the second container 34B are deleted and replaced with hard links pointing to the files of the first container 34A. The deduplication module 32 queries the deduplication file list based on the file-to-memory mapping information in the container snapshot to see whether the file corresponding to the non-anonymous memory page is marked as merged, calculates hash values ​​for the anonymous memory page and the non-anonymous memory page corresponding to the file not marked as merged, and records them in the deduplication memory list; the deduplication module 32 retains one copy of the memory pages with the same hash value in the deduplication memory list and the memory pages whose corresponding files are marked as merged, records the location of the copy, and deletes the remaining copies; without loss of generality, it is assumed that among the duplicate memory pages, the memory page of the first container 34A is retained and the memory page of the second container 34B is deleted. At this point, the present invention completes the deduplication operation on the container file system and the deduplication operation on the container memory.

[0082] For the deduplication-enhanced container processing function call requests, calling module 33 initiates a function call request to one of the containers. For the sake of generality, let's assume the first container 34A is the one to initiate the call. The snapshot file of first container 34A remains unchanged, so the container process is directly restored from the snapshot file to process the function call request. Calling module 33 then initiates a function call request to second container 34B. The memory page data in the snapshot file of second container 34B was altered during the deduplication operation. The deduplication memory list must be queried to find the only copy of the deleted memory page. After restoring the complete snapshot file, the container process is restored from the snapshot file to process the function call request. In this case, second container 34B shares files with first container 34A. When first container 34A starts, it opens the files required by second container 34B. These files are cached in the operating system's physical memory page cache. When second container 34B starts, it reads these files directly from the page cache and shares physical memory with first container 34A. This enables file and memory sharing between multiple containers on a single node, reducing storage and memory overhead and accelerating container startup.

[0083] In addition, the implementation process of the functions and effects of each module in the above system is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0084] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Those of ordinary skill in the art can understand and implement it without expending creative work.

[0085] Corresponding to the embodiment of the above-mentioned container resource occupancy overhead optimization method, the embodiment of the present application further provides a computer device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the following Figure 1 In addition, in addition to the processor, memory, DMA controller, disk, and non-volatile memory, the computer device may also include other hardware, depending on the actual function of the device, which will not be described in detail.

[0086] Corresponding to the embodiment of the aforementioned container resource occupancy overhead optimization method, an embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the container resource occupancy overhead optimization method in the above-mentioned embodiment. The computer-readable storage medium can be the internal storage unit of any device with data processing capabilities in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any device with data processing capabilities, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both the internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0087] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for optimizing container resource usage overhead, characterized in that: Include: Record the files accessed during the execution of the function process corresponding to each container and generate a file working set for each container; For a single container, only the files in the file working set are retained and other files are deleted. For multiple containers, duplicate files between different file working sets are identified for deduplication. Generate a snapshot file for each container and suspend the operation of the container, identify duplicate memory pages in each snapshot file for deduplication; When you need to start a new container, restore the container from the snapshot file.

2. The method according to claim 1, characterized in that For a file working set of a single container, redundant files outside the file working set are deleted to retain only the files within the file working set.

3. The method according to claim 1, characterized in that Deduplication of duplicate files between different file working sets. For duplicate files, only one copy is retained, and the remaining duplicate files are replaced with hard links pointing to the copy. The files that are replaced with hard links are recorded.

4. The method according to claim 3, characterized in that Identify duplicate files between different file working sets for multiple containers, including: Mark all files with the same path name in different file working sets as suspected duplicate files; For all suspected duplicate files, sample file data blocks of the same size at the same offset to obtain the sampled data blocks corresponding to each suspected duplicate file; A hash value is calculated for each sampled data block. If the hash values ​​of sampled data blocks of different suspected duplicate files are the same, the suspected duplicate file is confirmed as a duplicate file.

5. The method according to claim 1, characterized in that Deduplication of memory pages in snapshot files, including: For duplicate memory pages, only one copy is retained in one snapshot file, duplicate copies in other snapshot files are deleted, and the location of the retained copy is recorded.

6. The method according to claim 3, characterized in that Identify duplicate memory pages in each snapshot file, including: In the case where the memory pages in the snapshot file include anonymous memory pages, For anonymous memory pages, the hash value of each anonymous memory page is calculated, and the data of the anonymous memory pages with the same hash value are compared byte by byte. If the contents of two pages are exactly the same, they are identified as duplicate memory pages.

7. The method according to claim 3, characterized in that In the case where the memory pages in the snapshot file include non-anonymous memory pages, For a non-anonymous memory page, query whether a first file corresponding to the non-anonymous memory page is replaced with a hard link; If it has been replaced by a hard link, it is considered that the non-anonymous memory page corresponding to the first file is the same as the memory page corresponding to the second file pointed to by the hard link and has been merged, and the non-anonymous memory page is identified as a duplicate memory page; If it is not replaced by a hard link, the hash value of each non-anonymous memory page is calculated, and the data of the non-anonymous memory pages with the same hash value are compared byte by byte. If the contents of the two pages are exactly the same, they are identified as duplicate memory pages.

8. The method according to claim 7, characterized in that Store each snapshot file in memory, and each snapshot contains the mapping information from file to memory; It is queried based on the file-to-memory mapping information whether the first file corresponding to the non-anonymous memory page is replaced with a hard link.

9. The method according to claim 5, characterized in that Restoring the container from the snapshot file includes: For snapshot files that have not been modified during the deduplication phase, the container is restored directly from the snapshot file; For the snapshot file modified in the deduplication phase, the copy is read according to the location of the retained copy, and the initial snapshot file is restored and then the container is recovered from the initial snapshot file.

10. The method according to claim 1, characterized in that After receiving a function call request, the survival monitoring module in each container generates a function process to process the function call request and starts monitoring file access information of the function process; After the function process completes processing the function call request, all file access information of the function process throughout its life cycle is recorded to construct a working set of the file.

11. A container resource usage overhead optimization system, characterized in that: At least: A survival monitoring module records the files accessed during the execution of the function process corresponding to each container and generates a file working set for each container. The file deduplication module is used to retain only the files in the file working set of a single container and delete other files. It also identifies duplicate files between different file working sets of multiple containers for deduplication processing. A memory deduplication module, which generates a snapshot file for each container and pauses the container's operation, identifying duplicate memory pages in each snapshot file for deduplication processing; Call the module to restore the container from the snapshot file when a new container needs to be started.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the container resource occupancy overhead optimization method according to any one of claims 1 to 10.

13. A computer device, characterized in that: It includes a memory and a processor, the memory is coupled to the processor; wherein, at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the container resource occupancy overhead optimization method as described in any one of claims 1 to 10.

Images