A memory offloading method, system and device for container scenarios

Abstract

The application provides a memory unloading method, system and device for a container scenario, the method loads a strategy agent component on a control node of the machine cluster, loads a swap agent component and a swap index component in a computing node, three customized components can actively obtain the container instance that needs to be adjusted based on the number of container instances required by the current container service, and generate an adjustment parameter to adjust the occupied memory of the container instance. Compared with the prior art, the application greatly expands the use ability of the kubernetes container orchestration platform for swap memory, so that the kubernetes container orchestration platform changes from passively applying memory swap to avoid memory overrun to actively using the swap memory mechanism to save memory and improve cold start performance.

Description

Technical Field

[0001] This invention relates to the field of computer memory allocation, and more specifically, to a memory unloading method, system, and device for container scenarios. Background Technology

[0002] Against the backdrop of rapidly increasing computing power demand, cloud computing has gradually become the mainstream method of computing power supply. Container technology is a crucial cornerstone of the cloud computing field. With its lightweight, highly portable, and highly isolated characteristics, it has become a core component of modern cloud computing program development and deployment. The currently popular microservice architecture and functional computing approach are entirely based on container technology for the use and provision of cloud computing resources. In the fields of microservice architecture and functional computing, limited resources are a major constraint on container service deployment. Efficient resource utilization means being able to deploy more cloud services with limited cloud computing resources, generating greater economic benefits, or reducing energy consumption and providing greater environmental benefits while providing the same service capabilities. Even a slight increase in resource utilization of only one-thousandth can generate significant economic and environmental benefits when multiplied by the sheer volume of cloud computing resources. A considerable number of idle container instances are often deployed in cloud computing environments. These containers are not currently in use but occupy a large amount of cloud computing resources. Therefore, how to efficiently utilize the resources occupied by idle containers while ensuring service quality has become a pressing issue.

[0003] Considering that idle containers primarily occupy memory resources in cloud computing, the problem of conserving cloud computing resources in idle container scenarios can be largely transformed into a memory compression problem. The main goal is to efficiently utilize the memory occupied by idle containers while ensuring service quality. Currently, there are three main methods for handling idle containers: no additional processing, deletion and recycling, and replacement with child containers. However, these three methods still have the following shortcomings:

[0004] First, while not taking extra action against idle container instances ensures service capacity, it also means giving up the possibility of efficiently utilizing the memory they occupy. A large number of idle container instances will lead to high memory usage and low CPU utilization, wasting a lot of cloud computing resources, significantly reducing the economic benefits that the machine can generate, and violating the concept of efficient resource utilization.

[0005] Secondly, deleting and reclaiming idle container instances significantly reduces the service capacity of container services. Requests exceeding the service capacity will cause container instances to be recreated, and some services will bear the time overhead of container instance creation. This can easily lead to service time violations of the Service-Level Agreement (SLA). Furthermore, the frequent repetition of image pulling, instance container creation, and instance application initialization processes during container instance creation and deletion will also incur additional bandwidth and CPU overhead.

[0006] Third, recent research (Li Z, Guo L, Chen Q, et al. Help Rather Than Recycle: Alleviating Cold Startup in Serverless Computing Through Inter-Function Container Sharing[C] / / 2022 USENIX Annual Technical Conference(USENIX ATC 22).2022:69-84.) has proposed the concept of using sub-containers to replace idle containers. This involves packaging the runtime environments required by multiple idle container instances into a single sub-container image, creating this image in advance. When a request arrives, the corresponding instance application only needs to be imported into the sub-container to run, reducing the cold start time of container instances. However, this method requires a significant amount of CPU for combining and packaging the sub-container image, and the sub-container image also consumes a considerable amount of memory, indicating that the efficiency of memory resource utilization needs improvement. Summary of the Invention

[0007] The present invention aims to overcome at least one of the defects of the prior art and provide a memory offloading method for container scenarios, which can efficiently utilize the memory occupied by idle containers while ensuring service quality.

[0008] The technical solution adopted in this invention is as follows:

[0009] This invention provides a memory offloading method for container scenarios, applied to machine clusters based on the Linux operating system and the Kubernetes container orchestration platform. The method includes:

[0010] Enable the memory swapping function of the machine cluster; specifically, this is achieved by modifying the configuration of Linux and the kubelet component of Kubernetes.

[0011] A policy agent component is loaded in the control node of the machine cluster, and a switching agent component and a switching metric component are loaded in the compute nodes.

[0012] The number of container instances required by the current container service is obtained through the container instance number prediction algorithm of the machine cluster.

[0013] Based on the number of container instances required by the current container service, the operation of selecting the container instances according to the selection policy pre-configured in the policy proxy component, and adjusting the labels of the container instances;

[0014] The exchange proxy component monitors changes in the labels of the container instances, obtains container instances that have undergone preset changes, and processes the container instances through the exchange index component.

[0015] Once the exchange metrics component has finished processing the container instance, the exchange proxy component restores the corresponding label of the container instance to its pre-adjustment state.

[0016] By loading a policy agent component on the control node of the machine cluster, and a swap agent component and a swap metric component on the compute nodes, these three custom components can proactively obtain the container instances that need adjustment based on the number of container instances required by the current container service, and generate adjustment parameters to adjust the memory usage of the container instances. This greatly expands the Kubernetes container orchestration platform's ability to utilize swap memory, transforming it from passively applying memory swapping to avoid memory overruns to proactively utilizing swap memory mechanisms to save memory and improve performance. Simultaneously, tags are introduced to represent the state of the container instances. These tags can be used to transmit information about the container instances, allowing components to proactively obtain the container's state and make corresponding adjustments.

[0017] Furthermore, the tag includes a status tag, an exchange indication tag, and a storage occupancy tag;

[0018] The status label corresponds to the current running status of the container instance;

[0019] The swap indication label includes a first label value and a second label value. The swap indication label is set to the first label value to indicate that the container instance currently has an unexecuted memory adjustment request, and set to the second label value to indicate that the container instance currently does not have an unexecuted memory adjustment request.

[0020] The storage usage tag indicates the amount of memory required by the container instance.

[0021] Furthermore, the running states of the container instance include the created state, the running state, the sleeping state, and the terminated state;

[0022] The creation state refers to the state of the container instance after it is created and before it is used to provide container services.

[0023] The running state refers to the state in which the container instance is waiting for or responding to the container service request.

[0024] The sleep state is the state after the container instance is suspended;

[0025] The termination state is the state after the container instance is deleted.

[0026] The "creating state" refers to the state after a container is created but before it can provide services. After creation, the container instance enters the initialization process. Once initialization is complete, the container instance enters the "running state," where it can respond to various requests normally. When the service capacity of the container service exceeds the request load, the corresponding container instance is marked as needing to be suspended by a detection algorithm. The system will then transition the container instance from the running state to the sleeping state through a suspension action. At this time, most of the container instance's memory is swapped to the hard disk, and it enters the sleeping state. After entering the sleeping state, the container instance will no longer receive or respond to any requests, and in terms of service capacity, it can be considered deleted. When the service capacity of the container service is less than the request load, the corresponding container instance is marked as needing to be activated by a detection algorithm. The system will then transition the container instance from the sleeping state to the running state through an activation operation. At this time, the container instance's memory can be partially or completely swapped to the hard disk, and it enters the running state. After entering the running state, the container instance can respond to various requests normally, and in terms of service capacity, it is equivalent to creating a new container instance.

[0027] Furthermore, the step of selecting an operation on the container instance according to a pre-configured selection strategy in the policy proxy component, generating adjustment parameters for the container instance based on the selected operation, and adjusting the label of the container instance based on the adjustment parameters specifically includes:

[0028] The container instance is created / deleted or loaded / unloaded according to the selection policy pre-configured in the policy proxy component.

[0029] If you choose to create / delete the container instance, then the container instance will be operated according to the creation / deletion logic in the machine cluster;

[0030] If you choose to load / unload the container instance, the name of the container instance that needs to be adjusted, the adjusted running status, and the adjusted memory size will be generated.

[0031] The policy proxy component obtains the corresponding container instance based on the name of the container instance that needs to be adjusted, sets the status label of the container instance to the adjusted running state, sets the storage occupancy label to the adjusted memory size, and sets the swap indication label to the first label value.

[0032] The operation of selecting the container instance through the policy proxy component allows for selection based on the selection policy of the Kubernetes container orchestration platform on the machine cluster when creating or deleting the container instance, without conflicting with the Kubernetes system itself. Furthermore, this invention adds loading / unloading operations for the container instance. These loading / unloading operations are more lightweight than creation / deletion operations, effectively utilizing the container instance's resources. Reasonable use of these loading / unloading operations can reduce resource consumption and cold start overhead caused by frequent container creation / deletion.

[0033] Furthermore, obtaining the container instance that needs adjustment specifically includes:

[0034] Based on the selection strategy of the machine cluster, the container instance whose status label corresponds to the running state and whose exchange indicator label is set to the second label value is selected.

[0035] The two-layer selection mechanism of status labels and swap indication labels ensures that the container instance is in a normal working state. When the container instance is in the creation state, sleep state, terminated state, or needs to perform memory swapping operations, it will be removed from the service list of the Kubernetes container orchestration platform, and requests for that container service will be dispatched to the container instance in the running state. If no running container instance is currently available, a sleeping container instance will be activated first to respond to the request, as the overhead of activating the container instance is much less than creating and initializing a new container instance. If no running or sleeping container instances are available, a new container instance must be created and initialized.

[0036] Furthermore, the step of monitoring changes in the labels of the container instances through the exchange proxy component, obtaining the container instances that have undergone preset changes, and adjusting the memory usage of the container instances based on the changed labels through the exchange metric component specifically includes:

[0037] The exchange proxy component on each compute node monitors the label changes of the corresponding container instance;

[0038] When the exchange indicator tag value of the container instance is detected to change from the second tag value to the first tag value, the PID of the container instance is obtained through the exchange proxy component, and the PID and the storage usage tag are sent to the exchange indicator component.

[0039] The exchange metric component obtains the data structure of the container instance corresponding to the compute node based on the PID, and sets the memory usage field in the data structure according to the storage usage tag of the container instance.

[0040] The swap proxy component monitors changes in the labels of container instances. When a corresponding change is detected, the swap proxy component adjusts the memory usage field in the data structure to proactively perform memory swapping operations on the container instance.

[0041] Furthermore, once the memory usage field in the data structure is set, if it is necessary to actively load the memory usage pages of the container instance into memory, the swap proxy component will proactively access the memory usage pages of the container instance sequentially. By proactively accessing the memory usage pages of the container instance through the swap proxy component, the effect of pre-swapping in some or all of the memory usage pages can be achieved.

[0042] Furthermore, restoring the corresponding label of the container instance to its pre-adjustment state via the exchange proxy component specifically includes:

[0043] The exchange proxy component sets the exchange indicator label of the container instance to a second label.

[0044] Setting the exchange indicator label as the second label ensures the normal operation of subsequent operations of the container instance.

[0045] This invention also provides a memory offloading system for container scenarios, applied to machine clusters based on the Linux operating system and the Kubernetes container orchestration platform. The system includes:

[0046] The component loading module is used to load the policy agent component in the control node of the machine cluster, and to load the exchange agent component and the exchange metric component in the compute node.

[0047] The quantity prediction module is used to obtain the number of container instances required by the current container service through the container instance quantity prediction algorithm of the machine cluster.

[0048] The container adjustment module is used to select container instances according to the number of container instances required by the current container service, following a selection strategy pre-configured in the policy proxy component, and to adjust the labels of the container instances.

[0049] The monitoring and processing module is used to monitor changes in the labels of the container instances through the exchange proxy component, obtain the container instances that have undergone preset changes, and process the container instances through the exchange index component.

[0050] The container adjustment module is also used to restore the corresponding label of the container instance to its pre-adjustment state through the exchange proxy component after the exchange index component has completed the processing of the container instance.

[0051] The present invention also provides a computer device, including a memory and a processor, characterized in that the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the memory offloading method for container scenarios described above.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] 1. This invention sets up three custom components in the corresponding nodes of the machine cluster: a policy proxy component, a swap proxy component, and a swap metric component. These three custom components can proactively obtain the container instances that need adjustment based on the number of container instances required by the current container service, and generate adjustment parameters to adjust the memory usage of the container instances. This greatly expands the Kubernetes container orchestration platform's ability to use swap memory, enabling the Kubernetes container orchestration platform to proactively utilize the swap memory mechanism to save memory and improve performance, instead of passively applying memory swapping to avoid memory overruns.

[0054] 2. This invention adds tags to the container instance, including a status tag, a swap indicator tag, and a storage occupancy tag. The status tag, swap indicator tag, and storage occupancy tag can be used to transmit information between custom components to indicate the status of the container instance, making it convenient for the custom components to perform memory swapping operations based on the tags. Attached Figure Description

[0055] Figure 1 This is a flowchart of the memory unloading method of the present invention.

[0056] Figure 2 This is a diagram showing the operational state transitions of a container instance according to the present invention.

[0057] Figure 3 This is a system architecture diagram of the memory unloading system of the present invention.

[0058] The attached diagram is labeled as follows: Component loading module 1, quantity prediction module 2, container adjustment module 3, and listening processing module 4. Detailed Implementation

[0059] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the invention. To better illustrate the following embodiments, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0060] Example 1

[0061] like Figure 1 As shown, this embodiment provides a memory offloading method for container scenarios, which can be applied to machine clusters based on the Linux operating system and the Kubernetes container orchestration platform; preferably, it is applied to machine clusters built on a Linux kernel operating system with kernel version 3.11 or above and a Kubernetes container orchestration platform with version 1.22 or above; the memory offloading method of this embodiment sets three custom components in the corresponding nodes of the machine cluster: a strategy agent component, a swap agent component, and a swap indicator component. The three custom components can actively obtain the container instances that need to be adjusted based on the number of container instances required by the current container service, and generate adjustment parameters to adjust the memory occupied by the container instances. This greatly expands the Kubernetes container orchestration platform's ability to use swap memory, enabling the Kubernetes container orchestration platform to actively use the swap memory mechanism to save memory and improve performance instead of passively applying memory swapping to avoid memory overruns.

[0062] Specifically, the memory unloading method includes:

[0063] S1: Enable the swap memory function of the machine cluster;

[0064] This step can be achieved by modifying the configuration of Linux and the kubelet component of Kubernetes;

[0065] Specifically, in this embodiment, modifying the Linux configuration includes:

[0066] Configure the GRUB_CMDLINE_LINUX_DEFAULT settings in the Linux kernel's GRUB configuration, set zswap.enabled to 1, and configure the methods for zswap.compressor and zswap.zpool;

[0067] Modifying the configuration of the Kubernetes kubelet component specifically includes:

[0068] Set nodeSwap to true in the feature-gates of the kubelet component, set fail-swap-on to false, and set swapBehavior in the memorySwap field to UnlimitedSwap;

[0069] By modifying the configuration of Linux and the kubelet component of Kubernetes, we ensured that the Kubernetes container orchestration platform allowed the swap memory mechanism to function properly.

[0070] S2: Load the policy agent component in the control node of the machine cluster, and load the exchange agent component and the exchange index component in the computing node;

[0071] Specifically, in this embodiment, the policy proxy component and the exchange proxy component are written in Go and primarily rely on the Kubernetes container orchestration platform's client-go to manage the machine cluster. The exchange metrics component is written in C and is mainly used to adjust the memory usage (memory.high) parameter in the kernel cgroup data structure of the container instance.

[0072] The loading policy agent component in the control node and the switching agent component in the compute node are configured as daemon threads in the Linux system, configured to start after the network module of the machine cluster, and configured to start on boot and restart on failure. Taking advantage of the hot-swappable module feature of the Linux system, the switching indicator component in the compute node is compiled into a kernel module. Then, the insmod command of the Linux system is used to insert the kernel module into the Linux system, and the mknod command is used to register the kernel module as a special file. The special file can be set to " / dev / swap-indicator".

[0073] S3: Obtain the number of container instances required by the current container service using the container instance count prediction algorithm of the machine cluster;

[0074] Specifically, in this embodiment, the Kubernetes Extended HPA (Horizontal PodAutoscaler) algorithm can be used as the algorithm for predicting the number of container instances, performing reactive scaling operations based on the resources used by the container instances. The Extended HPA algorithm uses a RESTful request to pass the number of container instances required by the container service to the policy proxy component located on the control node. Simultaneously, in this embodiment, the Label mechanism of the Kubernetes container orchestration platform can be used for communication, and the Label mechanism is used to pass information to the exchange proxy component of the compute node.

[0075] In a preferred embodiment, the Kubernetes container orchestration platform can be built in conjunction with the OpenFaaS function management platform. OpenFaaS is a Kubernetes-based function management platform that adds components such as Gateway and Prometheus to the Kubernetes architecture, enabling the deployment and management of functional container instances. Since OpenFaaS has a function instance count control mechanism, the aforementioned extended HPA algorithm is replaced by OpenFaaS's function instance count control strategy as the container instance count prediction algorithm. The Gateway component in OpenFaaS sends a RESTful request to the policy proxy component located on the control node.

[0076] When the current container service of the machine cluster needs to adjust the memory usage of the container instances, the number of container instances required by the container service is predicted by running a quantity prediction algorithm. Based on the number of container instances, this embodiment can proactively adjust the memory usage of the container instances.

[0077] S4: Based on the number of container instances required by the current container service, select the container instances according to the selection policy pre-configured in the policy proxy component, and adjust the labels of the container instances;

[0078] Specifically, in this embodiment, the container instance is equipped with tags, including a status tag, an exchange indicator tag, and a storage occupancy tag;

[0079] The status label corresponds to the current running state of the container instance; specifically, the value of the status label can be set to creating, running, sleeping and terminated, which respectively correspond to the four running states of the container instance in this embodiment: creation state, running state, sleeping state and terminated state.

[0080] As described above, the running states of the container instance in this embodiment include the creation state, the running state, the sleeping state, and the terminated state;

[0081] Specifically, such as Figure 2 As shown, the "creation state" refers to the state after a container is created but before it can provide services. After creation, the container instance enters the initialization process. Once initialization is complete, the container instance enters the "running state," where it can respond to various requests normally. When the service capacity of the container service exceeds the request load, the corresponding container instance is marked as needing to be suspended by a detection algorithm. The system will then transition the container instance from the running state to the "sleep state" through a suspension action. At this time, most of the container instance's memory is swapped to the hard disk, and it enters the sleep state. After entering the sleep state, the container instance will no longer receive or respond to any requests, and in terms of service capacity, it can be considered deleted. When the service capacity of the container service is less than the request load, the corresponding container instance is marked as needing to be activated by a detection algorithm. The system will then transition the container instance from the sleep state to the running state through an activation operation. At this time, the container instance's memory can be partially or completely swapped to the hard disk, and it enters the running state. After entering the running state, the container instance can respond to various requests normally, and in terms of service capacity, it is equivalent to creating a new container instance.

[0082] The swap indicator label includes a first label value and a second label value. Specifically, the first label value can be set to "yes" and the second label value can be set to "no". When the swap indicator label is set to the first label value "yes", it indicates that the container instance has an unexecuted memory adjustment request, and the container instance is in the process of transitioning from the running state. Conversely, when the swap indicator label is set to the second label value "no", it indicates that the container instance does not have an unexecuted memory adjustment request, and the container instance is in a stable state.

[0083] In this embodiment, the current state of the container instance can be represented by the status label and the exchange indicator label, indicating whether it needs to undergo the transition of the running state;

[0084] The storage usage tag indicates the amount of memory required by the container instance. Specifically, the storage usage tag is an integer, indicating the amount of memory the container instance should currently use. The storage usage tag cannot be set to 0, as setting it to 0 can easily lead to memory oscillation, i.e., frequent swapping in and out of memory.

[0085] This step also includes the following steps:

[0086] S41: Based on the number of container instances currently required by the container service, the strategy proxy component determines whether to create / delete or load / unload the container instances according to the selection strategy pre-configured in the strategy proxy component. Specifically, the strategy proxy component uses the Kubernetes container editing platform's client-go to access and manage the machine cluster, and uses the Go language's net / http library to listen to a specific port of the control node, responding to RESTful requests sent to that port, mainly responding to POST requests sent to the / svc_update method. The POST request contains the name of the container service and the number of container instances currently required by the container service. The strategy proxy component obtains the number of container instances currently required by the container service through listening, and determines the operation of the container instances according to the selection strategy pre-configured in the strategy proxy component.

[0087] S42: If you choose to create / delete the container instance, then operate the container instance according to the creation / deletion logic in the machine cluster; S43: If you choose to load / unload the container instance, then generate the name of the container instance that needs to be adjusted, the adjusted running status, and the adjusted memory size required;

[0088] Normally, when a container instance is in the creation, sleep, or terminated state, or when a memory swapping operation is required, it will be removed from the selection list of the corresponding service in the Kubernetes container orchestration platform. Requests for that container service will then be dispatched to a container instance in the running state. If no running container instance is currently available, a sleeping container instance will be activated first to respond to the request, as activating a sleeping container instance is far less expensive than creating and initializing a new one. If no running or sleeping container instances are available, a new container instance must be created and initialized.

[0089] Therefore, in order to reduce the overhead of creating and initializing a new container instance, this embodiment adds the loading / unloading operation; and, while ensuring memory availability, the selection strategy pre-configured in the policy proxy component prioritizes the loading / unloading operation.

[0090] Specifically, the unloading operation involves swapping a portion of the container instance's memory usage to the hard disk, reducing the memory usage of the idle container instance. After being unloaded, the container instance transitions from the running state to the sleeping state. The loading operation involves swapping memory from the hard disk to the container instance when the memory usage of the container instance does not meet the current needs of the container service, restoring the performance of the container instance. After being loaded, the container instance transitions from the sleeping state to the running state. That is, the loading / unloading operation achieves the switching between the running state and the sleeping state of the container instance. Compared to the deletion operation, the unloading operation does not completely release the container instance. The Kubernetes container orchestration platform can choose to perform the unloading operation on the container instance. When the total container service capacity of the service cannot meet the load request, the container instance is loaded first, and then the container instance is transitioned from the sleeping state to the running state to complete the container service, greatly improving the cold start performance of the Kubernetes container orchestration platform.

[0091] Preferably, the policy proxy component can be configured with a container sleep pool of preset capacity; when selecting to uninstall a container instance, the container instance is preferentially added to the container sleep pool; when a new container instance needs to be added, and the remaining capacity of the container sleep pool cannot meet the memory usage of the container instance, a certain number of container instances added to the container sleep pool first are deleted according to the order in which they were added; when the Kubernetes container orchestration platform needs to select a container instance to complete the container service, it preferentially selects from the container sleep pool.

[0092] If the adjusted memory size exceeds the available memory size of the corresponding region, the excess memory needs to be unloaded to the hard disk. This operation can be performed automatically by kswapd in the Linux system.

[0093] In addition, in this embodiment, when the Selector of the service in the Kubernetes container editing platform selects the container instance that needs to be adjusted, it needs to select the container instance whose status label corresponds to the running state and whose exchange indicator label is set to the second label value, based on the original selection strategy; through the two-layer selection mechanism of status label and exchange indicator label, it is ensured that the container instance is in a normal working state;

[0094] In this embodiment, various types of container instances were used for testing. The test results show that, in cold start scenarios, the loading operation reduces cold start latency by an average of 63.11% compared to the creation operation. Actual testing showed that setting the available memory during suspension to 10MB reduced cold start latency by over 90% compared to the creation operation.

[0095] S44: The policy proxy component obtains the corresponding container instance according to the name of the container instance that needs to be adjusted, sets the status label of the container instance to the adjusted running state, sets the storage occupation label to the adjusted memory size to be occupied, and sets the swap indication label to the first label value.

[0096] S5: Listen to the changes in the labels of the container instances through the exchange proxy component, obtain the container instances that have undergone preset changes, and process the container instances through the exchange index component;

[0097] This step also includes the following steps:

[0098] S51: The exchange proxy component on each compute node listens for changes in the labels of the corresponding container instances; specifically, the exchange proxy component on each compute node uses the Informer module in client-go of the Kubernetes container editing platform to listen for changes in the labels of the pods belonging to the listening namespace on the corresponding compute node.

[0099] S52: When the exchange indicator tag value of the container instance is detected to change from the second tag value to the first tag value, the PID of the container instance is obtained through the exchange proxy component, and the PID and the storage occupancy tag are sent to the exchange indicator component.

[0100] Specifically, when the exchange indicator tag of the container instance in the corresponding pod changes from the second tag value to the first tag value, the exchange proxy component adds a container available memory change event associated with the detected container instance to its work queue, and processes all container available memory change events in the order they are added to the queue.

[0101] When processing the container available memory change event, the exchange proxy component obtains the PID of the associated container instance. Since the PID of the container instance cannot be obtained directly, the exchange proxy component can convert the name of the corresponding container instance into a PID through the runtime interface of the container instance and then send the PID to the exchange metrics component.

[0102] In this embodiment, when containerd is used as a container instance, the crictl tool can be used to extract the number in the container instance ID to match the PID of the container instance.

[0103] Additionally, the swap indicator component, as a kernel module of the compute node, can register a special file using the `mknod` command. This special file can be set to " / dev / swap-indicator". The swap agent component can communicate with the swap indicator component by writing parameters to this special file. Specifically, in this step, after obtaining the PID of the container instance, the swap agent component transmits the information to the swap indicator component by writing the PID and the storage occupancy tag to the special file.

[0104] S53: The exchange index component obtains the data structure of the container instance corresponding to the computing node according to the PID, and sets the memory usage field in the data structure according to the storage usage tag of the container instance.

[0105] Specifically, after obtaining the PID and storage usage tag, the swapping metric component searches for and retrieves the cgroup data structure of the container instance based on the PID, and sets the memory usage (memory.high) field in the cgroup data structure according to the storage usage tag. By setting and adjusting the memory usage field of the container instance's cgroup data structure, the active memory swapping operation of the container instance is realized. After the memory usage field is set, if the actual memory usage value is greater than the set value, kswapd in the Linux system will automatically unload the memory exceeding the limit to the hard disk.

[0106] Furthermore, once the memory usage field in the data structure is set, if it is necessary to actively load the memory usage pages of the container instance into memory, the swap proxy component will proactively access the memory usage pages of the container instance sequentially. By proactively accessing the memory usage pages of the container instance through the swap proxy component, the effect of pre-swapping in some or all of the memory usage pages can be achieved. By adding this proactive access operation, and in conjunction with an algorithm for predicting the number of container instances, the cold start time of the container instances can be further reduced.

[0107] In this embodiment, when a container instance is selected to be loaded, the value of the memory occupied field can be set to the memory limit value set by the container instance; when the container instance is selected to be swapped out, the value of the memory occupied field can be set to 2MB. In a system where the page size of the memory occupied page is 4KB, the container instance contains 512 memory occupied pages.

[0108] S6: When the exchange index component completes the processing of the container instance, the exchange proxy component restores the corresponding label of the container instance to its state before adjustment.

[0109] Specifically, in this step, when the swap indicator component completes the processing of the container instance, the swap proxy component changes the swap indicator tag of the container instance from the first tag value to the second tag value, indicating that the swap memory operation of the container instance has been completed, and at the same time, the corresponding container available memory change event is removed from the work queue of the swap proxy component.

[0110] Example 2

[0111] like Figure 3 As shown, this embodiment provides a memory offloading system for container scenarios, applied to machine clusters based on the Linux operating system and the Kubernetes container orchestration platform; preferably, it is applied to machine clusters built on a Linux kernel operating system with kernel version 3.11 or higher and a Kubernetes container orchestration platform with version 1.22 or higher; the memory offloading system of this embodiment sets three custom components in the corresponding nodes of the machine cluster: a strategy agent component, a swap agent component, and a swap indicator component. The three custom components can actively obtain the container instances that need to be adjusted based on the number of container instances required by the current container service, and generate adjustment parameters to adjust the memory occupied by the container instances. This greatly expands the Kubernetes container orchestration platform's ability to use swap memory, enabling the Kubernetes container orchestration platform to actively use the swap memory mechanism to save memory and improve performance instead of passively applying memory swapping to avoid memory overruns.

[0112] Before starting the memory offloading system described in this embodiment, it is necessary to enable the swap memory function of the machine cluster, which can be achieved by modifying the configuration of Linux and the configuration of the kubelet component of Kubernetes.

[0113] Specifically, in this embodiment, modifying the Linux configuration includes:

[0114] Configure the GRUB_CMDLINE_LINUX_DEFAULT settings in the Linux kernel's GRUB configuration, set zswap.enabled to 1, and configure the methods for zswap.compressor and zswap.zpool;

[0115] Modifying the configuration of the Kubernetes kubelet component specifically includes:

[0116] Set nodeSwap to true in the feature-gates of the kubelet component, set fail-swap-on to false, and set swapBehavior in the memorySwap field to UnlimitedSwap;

[0117] By modifying the configuration of Linux and the kubelet component of Kubernetes, we ensured that the Kubernetes container orchestration platform allowed the swap memory mechanism to function properly.

[0118] Furthermore, the memory offloading system includes:

[0119] Component loading module 1 is used to load a policy agent component in the control node of the machine cluster, and to load a switching agent component and a switching metric component in the compute node.

[0120] Specifically, in this embodiment, the policy proxy component and the exchange proxy component are written in Go and primarily rely on the Kubernetes container orchestration platform's client-go to manage the machine cluster. The exchange metrics component is written in C and is mainly used to adjust the memory usage (memory.high) parameter in the kernel cgroup data structure of the container instance.

[0121] The component loading module 1 configures the loading policy agent component in the control node and the switching agent component in the compute node as daemon threads in the Linux system, configures them to start after the network module of the machine cluster, and sets boot-up and failure restart policies for them; and utilizes the hot-swappable module feature of the Linux system to compile the switching indicator component in the compute node into a kernel module, then uses the insmod command of the Linux system to insert the kernel module into the Linux system, and uses the mknod command to register the kernel module as a special file, which can be set to " / dev / swap-indicator";

[0122] The quantity prediction module 2 is used to obtain the number of container instances required by the current container service through the container instance quantity prediction algorithm of the machine cluster.

[0123] Specifically, in this embodiment, the Kubernetes Extended HPA (Horizontal PodAutoscaler) algorithm can be used as the algorithm for predicting the number of container instances, performing reactive scaling operations based on the resources used by the container instances. The Extended HPA algorithm uses a RESTful request to pass the number of container instances required by the container service to the policy proxy component located on the control node. Simultaneously, in this embodiment, the Label mechanism of the Kubernetes container orchestration platform can be used for communication, and the Label mechanism is used to pass information to the exchange proxy component of the compute node.

[0124] In a preferred embodiment, the Kubernetes container orchestration platform can be built in conjunction with the OpenFaaS function management platform. OpenFaaS is a Kubernetes-based function management platform that adds components such as Gateway and Prometheus to the Kubernetes architecture, enabling the deployment and management of functional container instances. Since OpenFaaS has a function instance count control mechanism, the aforementioned extended HPA algorithm is replaced by OpenFaaS's function instance count control strategy as the container instance count prediction algorithm. The Gateway component in OpenFaaS sends a RESTful request to the policy proxy component located on the control node.

[0125] When the current container service of the machine cluster needs to adjust the memory usage of the container instances, the quantity prediction module 2 predicts the number of container instances required by the container service by running a quantity prediction algorithm. Based on the number of container instances, this embodiment can proactively adjust the memory usage of the container instances.

[0126] The container adjustment module 3 is used to select the container instance according to the selection policy pre-configured in the policy proxy component based on the number of container instances required by the current container service, and to adjust the label of the container instance.

[0127] Specifically, in this embodiment, the container instance is equipped with tags, including a status tag, an exchange indicator tag, and a storage occupancy tag;

[0128] The status label corresponds to the current running state of the container instance; specifically, the value of the status label can be set to creating, running, sleeping and terminated, which respectively correspond to the four running states of the container instance in this embodiment: creation state, running state, sleeping state and terminated state.

[0129] As described above, the running states of the container instance in this embodiment include the creation state, the running state, the sleeping state, and the terminated state;

[0130] Specifically, such as Figure 2 As shown, the "creation state" refers to the state after a container is created but before it can provide services. After creation, the container instance enters the initialization process. Once initialization is complete, the container instance enters the "running state," where it can respond to various requests normally. When the service capacity of the container service exceeds the request load, the corresponding container instance is marked as needing to be suspended by a detection algorithm. The system will then transition the container instance from the running state to the "sleep state" through a suspension action. At this time, most of the container instance's memory is swapped to the hard disk, and it enters the sleep state. After entering the sleep state, the container instance will no longer receive or respond to any requests, and in terms of service capacity, it can be considered deleted. When the service capacity of the container service is less than the request load, the corresponding container instance is marked as needing to be activated by a detection algorithm. The system will then transition the container instance from the sleep state to the running state through an activation operation. At this time, the container instance's memory can be partially or completely swapped to the hard disk, and it enters the running state. After entering the running state, the container instance can respond to various requests normally, and in terms of service capacity, it is equivalent to creating a new container instance.

[0131] The swap indicator label includes a first label value and a second label value. Specifically, the first label value can be set to "yes" and the second label value can be set to "no". When the swap indicator label is set to the first label value "yes", it indicates that the container instance has an unexecuted memory adjustment request, and the container instance is in the process of transitioning from the running state. Conversely, when the swap indicator label is set to the second label value "no", it indicates that the container instance does not have an unexecuted memory adjustment request, and the container instance is in a stable state.

[0132] In this embodiment, the current state of the container instance can be represented by the status label and the exchange indicator label, indicating whether it needs to undergo the transition of the running state;

[0133] The storage usage tag indicates the amount of memory required by the container instance. Specifically, the storage usage tag is an integer, indicating the amount of memory the container instance should currently use. The storage usage tag cannot be set to 0, as setting it to 0 can easily lead to memory oscillation, i.e., frequent swapping in and out of memory.

[0134] Specifically, the operation of selecting the container instance according to the selection policy pre-configured in the policy proxy component and adjusting the label of the container instance includes the following steps:

[0135] S41: Container adjustment module 3, based on the number of container instances currently required by the container service, determines whether to create / delete or load / unload the container instances according to the selection strategy pre-configured in the policy proxy component. Specifically, the policy proxy component uses the Kubernetes container editing platform's client-go to access and manage the machine cluster, and uses the Go language's net / http library to listen to a specific port of the control node, responding to RESTful requests sent to that port, primarily responding to POST requests sent to the / svc_update method. The POST request contains the name of the container service and the number of container instances currently required by the container service. The policy proxy component, by listening to and obtaining the number of container instances currently required by the container service, determines the operation of the container instances according to the selection strategy pre-configured in the policy proxy component.

[0136] S42: If you choose to create / delete the container instance, then operate the container instance according to the creation / deletion logic in the machine cluster;

[0137] S43: If you choose to load / unload the container instance, then generate the name of the container instance that needs to be adjusted, the adjusted running status, and the adjusted memory size.

[0138] Normally, when a container instance is in the creation, sleep, or terminated state, or when a memory swapping operation is required, it is removed from the selection list of the corresponding service in the Kubernetes container orchestration platform. Requests for that container service are then dispatched to a container instance in the running state. If no running container instance is currently available, a sleeping container instance is activated first to respond to the request, as activating a sleeping instance is far less expensive than creating and initializing a new one. If neither a running nor a sleeping container instance is available, a new container instance must be created and initialized.

[0139] Therefore, in order to reduce the overhead of creating and initializing a new container instance, this embodiment adds the loading / unloading operation; and, while ensuring memory availability, the selection strategy pre-configured in the policy proxy component prioritizes the loading / unloading operation.

[0140] Specifically, the unloading operation involves swapping a portion of the container instance's memory usage to the hard disk, reducing the memory usage of the idle container instance. After being unloaded, the container instance transitions from the running state to the sleeping state. The loading operation involves swapping memory from the hard disk to the container instance when the memory usage of the container instance does not meet the current needs of the container service, restoring the performance of the container instance. After being loaded, the container instance transitions from the sleeping state to the running state. That is, the loading / unloading operation achieves the switching between the running state and the sleeping state of the container instance. Compared to the deletion operation, the unloading operation does not completely release the container instance. The Kubernetes container orchestration platform can choose to perform the unloading operation on the container instance. When the total container service capacity of the service cannot meet the load request, the container instance is loaded first, and then the container instance is transitioned from the sleeping state to the running state to complete the container service, greatly improving the cold start performance of the Kubernetes container orchestration platform.

[0141] Preferably, the policy proxy component can be configured with a container sleep pool of preset capacity; when selecting to uninstall a container instance, the container instance is preferentially added to the container sleep pool; when a new container instance needs to be added, and the remaining capacity of the container sleep pool cannot meet the memory usage of the container instance, a certain number of container instances added to the container sleep pool first are deleted according to the order in which they were added; when the Kubernetes container orchestration platform needs to select a container instance to complete the container service, it can select one from the container sleep pool.

[0142] If the adjusted memory size exceeds the available memory size of the corresponding region, the excess memory needs to be unloaded to the hard disk. This operation can be performed automatically by kswapd in the Linux system.

[0143] In addition, in this embodiment, when the Selector of the service in the Kubernetes container editing platform selects the container instance that needs to be adjusted, it needs to select the container instance whose status label corresponds to the running state and whose exchange indicator label is set to the second label value, based on the original selection strategy.

[0144] The two-layer selection mechanism of status label and exchange indicator label ensures that the container instance is in a normal working state;

[0145] In this embodiment, various types of container instances were used for testing. The test results show that, in cold start scenarios, the loading operation reduces cold start latency by an average of 63.11% compared to the creation operation. Actual testing showed that setting the available memory during suspension to 10MB reduced cold start latency by over 90% compared to the creation operation.

[0146] S44: The policy proxy component obtains the corresponding container instance according to the name of the container instance that needs to be adjusted, sets the status label of the container instance to the adjusted running state, sets the storage occupation label to the adjusted memory size to be occupied, and sets the swap indication label to the first label value.

[0147] The listening and processing module 4 is used to listen to the changes in the labels of the container instance through the exchange proxy component, obtain the container instance that has undergone a preset change, and process the container instance through the exchange index component.

[0148] Specifically, the processing by the monitoring module 4 includes the following steps:

[0149] S51: The exchange proxy component on each compute node listens for changes in the labels of the corresponding container instances; specifically, the exchange proxy component on each compute node uses the Informer module in client-go of the Kubernetes container editing platform to listen for changes in the labels of the pods belonging to the listening namespace on the corresponding compute node.

[0150] S52: When the exchange indicator tag value of the container instance is detected to change from the second tag value to the first tag value, the PID of the container instance is obtained through the exchange proxy component, and the PID and the storage occupancy tag are sent to the exchange indicator component.

[0151] Specifically, when the exchange indicator tag of the container instance in the corresponding pod changes from the second tag value to the first tag value, the exchange proxy component adds a container available memory change event associated with the detected container instance to its work queue, and processes all container available memory change events in the order they are added to the queue.

[0152] When processing the container available memory change event, the exchange proxy component obtains the PID of the associated container instance. Since the PID of the container instance cannot be obtained directly, the exchange proxy component can convert the name of the corresponding container instance into a PID through the runtime interface of the container instance and then send the PID to the exchange metrics component.

[0153] In this embodiment, when containerd is used as a container instance, the crictl tool can be used to extract the number in the container instance ID to match the PID of the container instance.

[0154] Furthermore, the swap indicator component, as a kernel module of the compute node, can register a special file using the `mknod` command. This special file can be set to " / dev / swap-indicator". The swap agent component can communicate with the swap indicator component by writing parameters to this special file. Specifically, after obtaining the PID of the container instance, the swap agent component transmits the information to the swap indicator component by writing the PID and the storage usage tag to the special file.

[0155] S53: The exchange index component obtains the data structure of the container instance corresponding to the computing node according to the PID, and sets the memory usage field in the data structure according to the storage usage tag of the container instance.

[0156] Specifically, after obtaining the PID and storage usage tag, the swapping metric component searches for and retrieves the cgroup data structure of the container instance based on the PID, and sets the memory usage (memory.high) field in the cgroup data structure according to the storage usage tag. By setting and adjusting the memory usage field of the container instance's cgroup data structure, the active memory swapping operation of the container instance is realized. After the memory usage field is set, if the actual memory usage value is greater than the set value, kswapd in the Linux system will automatically unload the memory exceeding the limit to the hard disk.

[0157] Furthermore, once the memory usage field in the data structure is set, if it is necessary to actively load the memory usage pages of the container instance into memory, the swap proxy component will proactively access the memory usage pages of the container instance sequentially. By proactively accessing the memory usage pages of the container instance through the swap proxy component, the effect of pre-swapping in some or all of the memory usage pages can be achieved. By adding this proactive access operation, and in conjunction with an algorithm for predicting the number of container instances, the cold start time of the container instances can be further reduced.

[0158] In this embodiment, when a container instance is selected to be loaded, the value of the memory occupied field can be set to the memory limit value set by the container instance; when the container instance is selected to be swapped out, the value of the memory occupied field can be set to 2MB. In a system where the page size of the memory occupied page is 4KB, the container instance contains 512 memory occupied pages.

[0159] The container adjustment module 3 is also used to restore the corresponding label of the container instance to its pre-adjustment state through the exchange proxy component when the exchange index component completes the processing of the container instance.

[0160] Specifically, when the swap indicator component completes the processing of the container instance, the swap proxy component changes the swap indicator tag of the container instance from the first tag value to the second tag value, indicating that the swap memory operation of the container instance has been completed, and at the same time removes the corresponding container available memory change event from the work queue of the swap proxy component.

[0161] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A memory offloading method for a container scenario, applied to a machine cluster based on a Linux operating system and a Kubernetes container orchestration platform, characterized in that, The method includes: Enable the swap memory function of the machine cluster; A policy agent component is loaded in the control node of the machine cluster, and a switching agent component and a switching metric component are loaded in the compute nodes. The number of container instances required for the current container service is obtained through the container instance number prediction algorithm of the machine cluster. Based on the number of container instances required by the current container service, the operation on the container instance is selected according to the selection policy pre-configured in the policy proxy component, the adjustment parameters of the container instance are generated according to the selected operation, and the label of the container instance is adjusted according to the adjustment parameters. The tag includes a status tag, an exchange indication tag, and a storage occupancy tag; The status label corresponds to the current running status of the container instance; The swap indication label includes a first label value and a second label value. The swap indication label is set to the first label value to indicate that the container instance currently has an unexecuted memory adjustment request, and set to the second label value to indicate that the container instance currently does not have an unexecuted memory adjustment request. The storage usage tag indicates the amount of memory required by the container instance; The exchange proxy component monitors changes in the labels of the container instances, identifies container instances that have undergone preset changes, and adjusts the memory usage of the container instances based on the changed labels using the exchange metrics component. Once the exchange metrics component has finished processing the container instance, the exchange proxy component restores the corresponding label of the container instance to its pre-adjustment state. The process of selecting an operation for the container instance according to a pre-configured selection strategy in the policy proxy component, generating adjustment parameters for the container instance based on the selected operation, and adjusting the label of the container instance based on the adjustment parameters specifically includes: The container instance is created / deleted or loaded / unloaded according to the selection policy pre-configured in the policy proxy component. If you choose to create / delete the container instance, then the container instance will be operated according to the creation / deletion logic in the machine cluster; If you choose to load / unload the container instance, the name of the container instance that needs to be adjusted, the adjusted running status, and the adjusted memory size will be generated. The policy proxy component obtains the corresponding container instance based on the name of the container instance that needs to be adjusted, sets the status label of the container instance to the adjusted running state, sets the storage occupancy label to the adjusted memory size, and sets the swap indication label to the first label value.

2. The memory offloading method for container scenarios according to claim 1, characterized in that, The container instance's running states include creation state, running state, sleeping state, and terminated state; The creation state refers to the state of the container instance after it is created and before it is used to provide container services. The running state refers to the state in which the container instance is waiting for or responding to the container service request. The sleep state is the state after the container instance is suspended; The termination state is the state after the container instance is deleted.

3. A memory unloading method for container scenarios according to claim 2, characterized in that, When selecting the container instance that needs adjustment, based on the selection strategy of the machine cluster, the container instance whose status label corresponds to the running state and whose exchange indicator label is set to the second label value is selected.

4. The memory unloading method for container scenarios according to claim 1, characterized in that, The step of monitoring changes in the labels of container instances through the exchange proxy component, obtaining container instances that have undergone preset changes, and adjusting the memory usage of container instances based on the changed labels through the exchange metric component specifically includes: The exchange proxy component on each compute node monitors the label changes of the corresponding container instance; When the exchange indicator tag value of the container instance is detected to change from the second tag value to the first tag value, the PID of the container instance is obtained through the exchange proxy component, and the PID and the storage usage tag are sent to the exchange indicator component. The exchange metric component obtains the data structure of the container instance corresponding to the compute node based on the PID, and sets the memory usage field in the data structure according to the storage usage tag of the container instance.

5. A memory unloading method for container scenarios according to claim 4, characterized in that, Once the memory usage field in the data structure is set, if it is necessary to actively load the memory usage page of the container instance into memory, the memory usage page of the container instance will be actively accessed sequentially through the exchange proxy component.

6. A memory offloading method for container scenarios according to claim 1, characterized in that, The step of restoring the corresponding label of the container instance to its pre-adjustment state through the exchange proxy component specifically includes: The exchange proxy component sets the exchange indicator label of the container instance to a second label.

7. A memory offloading system for container scenarios, applied to machine clusters based on the Linux operating system and the Kubernetes container orchestration platform, characterized in that, The system includes: The component loading module is used to load the policy agent component in the control node of the machine cluster, and to load the exchange agent component and the exchange metric component in the compute node. The quantity prediction module is used to obtain the number of container instances required by the current container service through the container instance quantity prediction algorithm of the machine cluster. The container adjustment module is used to select container instances according to a pre-configured selection policy in the policy proxy component based on the number of container instances currently required by the container service, and to adjust the labels of the container instances. The labels include a status label, a swap indication label, and a storage usage label. The status label corresponds to the current running status of the container instance. The swap indication label includes a first label value and a second label value. Setting the swap indication label to the first label value indicates that the container instance currently has unexecuted memory adjustment requests, and setting it to the second label value indicates that the container instance currently does not have unexecuted memory adjustment requests. The storage usage label indicates the amount of memory required by the container instance. The monitoring and processing module is used to monitor changes in the labels of the container instances through the exchange proxy component, obtain the container instances that have undergone preset changes, and process the container instances through the exchange index component. The container adjustment module is also used to restore the corresponding label of the container instance to its pre-adjustment state through the exchange proxy component when the exchange index component completes the processing of the container instance; The operation of selecting the container instance according to the pre-configured selection strategy in the policy proxy component and adjusting the label of the container instance specifically includes: The container instance is created / deleted or loaded / unloaded according to the selection policy pre-configured in the policy proxy component. If you choose to create / delete the container instance, then the container instance will be operated according to the creation / deletion logic in the machine cluster; If you choose to load / unload the container instance, the name of the container instance that needs to be adjusted, the adjusted running status, and the adjusted memory size will be generated. The policy proxy component obtains the corresponding container instance based on the name of the container instance that needs to be adjusted, sets the status label of the container instance to the adjusted running state, sets the storage occupancy label to the adjusted memory size, and sets the swap indication label to the first label value.

8. A computer device, comprising a memory and a processor, characterized in that, The memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the memory offloading method for container scenarios as described in any one of claims 1-6.

Images