Memory management method and device, equipment, storage medium, program product and chip

Abstract

The invention provides a memory management method and device, equipment, a storage medium, a program product and a chip. The method relates to the technical field of mobile terminals. Comprising the steps of starting an asynchronous compression process under the condition that a current idle memory is lower than a waterline value; and traversing the plurality of memory control groups, and allocating a thread to each memory control group through an asynchronous compression process so as to compress the anonymous page in each memory control group. According to the method provided by the invention, the compression efficiency is improved by asynchronizing the anonymous page compression part in the memory recovery process, creating the thread pool and performing parallel compression according to the memcg memory control group, and meanwhile, the asynchronous memory compression process does not influence the main recovery process.

Description

Technical Field

[0001] The present disclosure relates to the field of mobile terminal technology, and in particular to a memory management method and apparatus, equipment, storage medium, program product and chip. Background Art

[0002] With the variety of applications in mobile terminals, the memory usage of applications is gradually increasing, and the resulting system memory pressure is gradually increasing, causing the device to freeze due to insufficient memory resources, greatly affecting the user experience. At present, memory compression technology is used to save memory by identifying relatively cold memory pages during system operation, but the memory compression process often has problems such as high load and long time consumption. The recycling thread caused by it occupies the core for a long time, which can easily cause system freezes. At the same time, due to the long compression time, it will block the entire recycling process, thereby affecting the efficiency of memory recycling. Summary of the invention

[0003] The present disclosure provides a memory management method and apparatus, equipment, storage medium, program product and chip to solve the problems of long compression time and low memory recycling efficiency.

[0004] The first aspect of the embodiment of the present disclosure proposes a memory management method, which includes: when the current free memory is lower than the waterline value, starting an asynchronous compression process; traversing multiple memory control groups, and assigning threads to each memory control group through the asynchronous compression process to compress anonymous pages in each memory control group.

[0005] In the above embodiment, by judging the current free memory, when it is lower than the waterline value, the asynchronous compression process is started, and for each memory control group, a thread is assigned to perform independent anonymous page compression processing, so that the anonymous page compression does not affect the memory recovery process of the main program, and the efficiency of memory recovery is further improved.

[0006] In some embodiments of the present disclosure, when the current free memory is lower than the watermark value, the method also includes at least one of the following: if the current first free memory is greater than the second watermark value and less than the first watermark value, start an asynchronous recovery process; if the current second free memory is less than or equal to the second watermark value, start a direct recovery process, the asynchronous recovery process and the direct recovery process are used to recycle file pages in each memory control group, and the second watermark value is less than the first watermark value.

[0007] In the above embodiment, the current free memory is compared with the first watermark value and the second watermark value to determine whether the recycling process is asynchronous recycling or direct recycling, so as to recycle the file pages for each memory control group.

[0008] In some embodiments of the present disclosure, multiple memory control groups are traversed, and threads are assigned to each memory control group through an asynchronous compression process to compress anonymous pages in each memory control group, including: determining a target compression amount for a first memory control group, and adding the first memory control group and the target compression amount to a task queue; when the task queue is not empty, determining a maximum number of threads in a thread pool based on a current first CPU load; and for a first task in the task queue, assigning a first thread based on the maximum number of threads to compress anonymous pages in the first memory control group.

[0009] In the above embodiment, the asynchronous compression process determines the compression amount for each memory control group, adds a task queue, determines the maximum number of threads that can be carried according to the current CPU load, and compresses the compression tasks in the task queue based on the maximum number of threads, thereby achieving high efficiency of asynchronous compression.

[0010] In some embodiments of the present disclosure, determining the maximum number of threads in a thread pool based on the current first CPU load includes: obtaining historical operating data and the number of CPUs, the historical operating data including the number of threads running in a first time period; determining the average number of threads based on the historical operating data and the number of CPUs; and determining the maximum number of threads based on the average number of threads.

[0011] In the above embodiment, based on the number of threads and the number of CPUs running in the historical time period, the maximum number of threads is determined by the average number of threads, so as to effectively control the resource consumption of the thread pool while ensuring high concurrency performance, make full use of CPU resources, and reduce resource waste.

[0012] In some embodiments of the present disclosure, the method also includes: when the first thread is executed, based on the current second CPU load, updating the maximum number of threads; if the updated maximum number of threads is greater than or equal to the maximum number of threads, allocating the second task in the task queue to the first thread, and / or allocating the third task to the added second thread.

[0013] In the above embodiment, the maximum number of threads is updated based on the real-time CPU load to compress the compression tasks in the task queue, thereby fully utilizing CPU resources, avoiding resource waste, and further improving compression efficiency.

[0014] In some embodiments of the present disclosure, the method also includes any of the following: when the average number of threads is less than a first threshold, the number of CPUs is determined as the maximum number of threads; when the average number of threads is greater than the first threshold and less than a second threshold, half of the number of CPUs is determined as the maximum number of threads; when the average number of threads is greater than the second threshold, the maximum number of threads is determined to be one, and the first threshold is less than the second threshold.

[0015] In the above embodiment, by setting the first threshold and the second threshold, the maximum number of threads is reasonably determined according to the average number of threads, so as to fully utilize CPU resources, avoid resource waste, and further improve compression efficiency.

[0016] In some embodiments of the present disclosure, the method further includes: if the updated maximum number of threads is less than the maximum number of threads, the first thread executes sleep; waits for the maximum number of threads to be updated or the first thread is destroyed.

[0017] In the above embodiment, based on the updated maximum number of threads, the states of the threads are dynamically adjusted to ensure the activity of the threads so that the created threads can be fully utilized.

[0018] The second aspect of the present disclosure proposes a memory management device, including: a first processing module, used to start an asynchronous compression process when the current free memory is lower than the waterline value; a second processing module, used to traverse multiple memory control groups, and assign threads to each memory control group through the asynchronous compression process to compress anonymous pages in each memory control group.

[0019] An embodiment of the third aspect of the present disclosure proposes an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when used to run the computer program, executes the method described in any one of the embodiments of the first aspect of the present disclosure.

[0020] The fourth aspect embodiment of the present disclosure proposes a computer storage medium, wherein the computer storage medium stores computer executable instructions; after the computer executable instructions are executed by a processor, the method described in any one of the embodiments of the first aspect of the present disclosure can be implemented.

[0021] A fifth aspect embodiment of the present disclosure provides a computer program product. When the computer program product is run on a computer, the computer executes the method described in any one of the embodiments of the first aspect of the present disclosure.

[0022] The sixth aspect embodiment of the present disclosure proposes a chip, comprising at least one processor and a communication interface; the communication interface is used to receive signals input into the chip or signals output from the chip, the processor communicates with the communication interface and implements the method described in any one of the embodiments of the first aspect of the present disclosure through logic circuits or execution code instructions.

[0023] In summary, the memory management method, device, equipment and system proposed in the present disclosure include: when the current free memory is lower than the waterline value, start the asynchronous compression process; traverse multiple memory control groups, and assign threads to each memory control group through the asynchronous compression process to compress the anonymous pages in each memory control group. The method provided by the present disclosure improves the compression efficiency by asynchronousizing the anonymous page compression part in the memory recovery process, creating a thread pool and performing parallel compression according to the memcg memory control group. At the same time, the asynchronous memory compression process does not affect the main recovery process. It solves the problem that the entire recovery process is blocked due to the long compression time of anonymous pages, thereby affecting the recovery efficiency.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0026] Figure 1 A method flow chart of a memory management method proposed in an embodiment of the present disclosure;

[0027] Figure 2 A flow chart of a method for compressing anonymous pages proposed in an embodiment of the present disclosure;

[0028] Figure 3 A flow chart of a method for determining the maximum number of threads proposed in an embodiment of the present disclosure;

[0029] Figure 4 A method flow chart of the memory management method proposed in the embodiment of the present disclosure;

[0030] Figure 5 A method flow chart of the memory management method proposed in the embodiment of the present disclosure;

[0031] Fig. 6A A flowchart of the memory management method provided by the present disclosure;

[0032] Figure 6B A schematic diagram of determining a memory recycling method provided by the present disclosure;

[0033] Figure 6C A schematic diagram of the asynchronous compression process provided by the present disclosure;

[0034] Fig.6D Schematic diagram of the allocation strategy provided for the present disclosure

[0035] Figure 7A schematic diagram of the structure of a memory management device proposed in an embodiment of the present disclosure;

[0036] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0038] The following is an explanation of the professional terms in the embodiments of the present disclosure:

[0039] 1. Asynchronous recycling: When physical memory is tight, the kswapd kernel thread will be awakened to recycle memory. This memory recycling process is asynchronous and will not block the execution of the process. 2. Direct recycling: If the background asynchronous recycling cannot keep up with the speed of the process memory application, direct recycling will begin. This memory recycling process is synchronous and will block the execution of the process.

[0040] 3. File page recycling: For clean pages, the memory is released directly. This operation will not affect performance. For dirty pages, they are written back to disk first and then the memory is released. This operation will affect system performance.

[0041] 4. Recycling of anonymous pages: If the Swap mechanism is enabled, the Swap mechanism will swap out infrequently accessed anonymous pages to disk, and then swap them from disk to memory the next time they are accessed. This operation will affect system performance.

[0042] 5. Memory control groups: Memory control groups, referred to as memcg, are used for memory accounting. Currently, the Linux kernel uses memory control groups to implement process-level memory accounting. A memcg instance can contain multiple processes. The memory usage of these processes is accounted and restricted by the memcg. memcg can be divided into different dimensions for different manufacturers, either by application, by application type, or by a single process, for precise management and control.

[0043] 6. kswapd kernel thread: The full name is kswapd%d, where %d represents the ID of the memory node. In the Linux system, the kswapd thread is responsible for recycling pages when memory is insufficient to improve the overall performance of the system. Each NUMA memory node creates a kernel thread named kswapd%d, where %d is the ID of the node.

[0044] 7. NUMA (Non-Uniform Memory Access) architecture is a memory design for multi-processor systems that aims to optimize the performance of processor access to memory. This architecture enables memory to be allocated to different nodes, each node has its own local memory. The processor can access local memory faster than accessing the memory of other nodes. The nodes communicate through a high-speed interconnection network, and when the number of CPUs is increased, it can be achieved by increasing the number of nodes without causing excessive pressure on the bus. This architecture improves the performance of multi-processor systems, especially when the number of processors increases, by optimizing memory access to reduce latency. It is convenient for system expansion and easy to increase the number of CPUs.

[0045] Figure 1 This is a flow chart of a memory management method proposed in an embodiment of the present disclosure. Figure 1 As shown, the following steps are included:

[0046] Step 101, when the current free memory is lower than the waterline value, start the asynchronous compression process.

[0047] In some embodiments, when the current free memory is lower than the waterline value, at least one of the following items is also included: if the current first free memory is greater than the second waterline value and less than the first waterline value, an asynchronous recovery process is started; if the current second free memory is less than or equal to the second waterline value, a direct recovery process is started, the second waterline value is less than the first waterline value, and the second waterline value is less than the first waterline value.

[0048] In some embodiments, the memory recovery process includes a direct recovery process and an asynchronous recovery process. When the current free memory is lower than the waterline value, the memory recovery process needs to be started to recycle the file pages and anonymous pages in the current memory so that the free memory rises above the waterline value to ensure that there is memory available for allocation to the application.

[0049] In some embodiments, when a new application requests memory, based on the current free memory and the preset watermark value, it is determined whether the current free memory meets the watermark value to determine whether a memory recovery process needs to be started, and whether the started memory recovery process is a direct recovery process or an asynchronous recovery process.

[0050] In some embodiments, the first watermark value is the threshold for starting an asynchronous recovery process. If the current first free memory is lower than the first watermark value, it means that the current free memory can no longer meet the kernel's memory allocation for the application, and the kswapd kernel thread will be awakened to reclaim memory. This memory recovery process is asynchronous and will not block the execution of the process. When the background asynchronous recovery process cannot keep up with the speed of memory application, resulting in a continuous decrease in free memory, and the current second free memory is lower than the second watermark value, a direct recovery process will be started. This memory recovery process is synchronous and will block the execution of the process.

[0051] In some embodiments, starting the memory recovery process includes calculating the target recovery amount, that is, determining the target recovery amount to be recovered based on the current free memory and waterline value to determine the amount of tasks that the memory recovery process needs to perform. When the memory recovery process completes the target recovery amount corresponding to the file page recovery, the file page is written back or demapped. Among them, the file page write-back can be to write the modified data page back to the corresponding disk file to maintain the consistency of the data on the memory and disk and reduce the pressure of memory usage; demapping can be to release the physical memory resources occupied by the memory page, or mark these resources as reusable to increase the memory usage space.

[0052] In some embodiments, when the current free memory is lower than the waterline value, the memory recovery process can be started by determining a target recovery amount based on the difference between the current free memory and the waterline value. When the recovery reaches the target recovery amount, the memory recovery process ends and it is determined whether the current free memory is lower than the waterline value. If not, the memory recovery process is not started. If it is still lower than the waterline value, the memory recovery process is started again.

[0053] For example, Figure 6B The schematic diagram of determining the memory recovery method is shown in FIG. Asynchronous recovery is performed by the recovery thread Kswapd in the system. When the system available memory is lower than the low watermark, the work is awakened to recover the memory to the high watermark. When the system enables the NUMA architecture, the memory will be divided into multiple nodes according to the CPU cluster. Each node will be configured with a Kswapdn (n corresponds to the node identifier) ​​thread to handle tasks such as memory release and compression of the node. Since the mobile terminal only enables one node due to hardware characteristics, all asynchronous recovery tasks fall into this thread. When the application cannot apply for memory due to tight system memory resources, the system available memory is lower than the min (minimum, minimum / lowest limit) watermark, and the application itself executes the memory recovery process, that is, the direct recovery process.

[0054] For example, when a new application applies for memory, determine whether it can directly apply for free memory. If it can directly apply for free memory, the application is completed and memory is allocated to the application. If it cannot directly apply for free memory, determine whether the current zone meets the low watermark. If it does, determine whether it meets the min watermark. If it does, determine again whether it can directly apply for free memory. If it does not meet the min watermark, execute the direct recovery process. If the current zone does not meet the low watermark, wake up kswapd and execute the asynchronous recovery process.

[0055] Step 102 , traverse multiple memory control groups, and assign a thread to each memory control group through an asynchronous compression process to perform compression processing on anonymous pages in each memory control group.

[0056] In some embodiments, traversing multiple memory control groups may be used to determine the reclaim ratio of file pages and anonymous pages that require memory reclaim.

[0057] In some embodiments, traversing multiple memory control groups may be to reclaim file pages in each memory reclaim group through a memory reclaim process, and to compress anonymous pages in each memory control group through an asynchronous compression process.

[0058] In some embodiments, traversing multiple memory control groups and reclaiming file pages in each memory recovery group through a memory recovery process can wake up a file page recovery thread and recycle file pages in memory, such as writing back and releasing file pages, to reduce the memory occupancy of file pages.

[0059] In some embodiments, traversing the memory control group may be traversing all memory control groups, and after extracting iru in each memory control group, calculating the recovery ratio of file pages and anonymous pages. Among them, iru corresponds to a variable, data structure or field defined in an application, tool or script for storing, tracking or representing a certain memory usage or performance indicator. In other words, there may be only file pages that need to be recycled, or only anonymous pages that need to be recycled, or both file pages and anonymous pages that need to be recycled. Then, by traversing the memory control group memcg and taking iru, the recovery ratio of file pages and anonymous pages that need to be recycled can be determined.

[0060] In some embodiments, the memory recovery process includes a direct recovery process and an asynchronous recovery process. When the watermark value of the asynchronous recovery process is met, the file pages are recovered through the asynchronous recovery process. When the watermark value of the direct recovery process is met, the file pages are recovered through the direct recovery process.

[0061] In some embodiments, multiple memory control groups are traversed, and anonymous pages in each memory control group are compressed through an asynchronous compression process. That is, no matter whether the recovery of the file page currently determined by traversing the memory control groups is processed by direct recovery or asynchronous recovery process, the anonymous page is compressed through the asynchronous process and executed in parallel with the memory recovery process to avoid the compression process affecting the efficiency of the main recovery process.

[0062] In some embodiments, traversing multiple memory control groups, assigning threads to each memory control group through an asynchronous compression process, and compressing anonymous pages in each memory control group can be an asynchronous compression process for anonymous pages according to the grouping of memory control groups. In other words, the anonymous page compression process is asynchronous with the main memory recovery process, and the compression of anonymous pages is processed according to each memory control group, that is, each memory control group will form an anonymous page compression task, and the asynchronous compression process assigns threads to each compression task for compression processing, so that the compression of anonymous pages is parallelized according to memcg groups, and the efficiency of compression and traversing memcg is improved.

[0063] In the above embodiment, when the current free memory reaches the preset waterline value, the memory recovery process is started. Furthermore, the file page recovery is handled by the memory recovery process, and the anonymous page compression is handled by the asynchronous compression process, so as to realize the complete asynchronous compression of the anonymous page, and perform it in parallel with the main recovery process to avoid affecting the main recovery process; and the anonymous page compression is processed according to the grouping of memcg, so as to further improve the speed of compression and traversal of memcg, realize the high efficiency of the memory management process, reduce the delay of the asynchronous recovery process and the data processing of the application, and do not block the application memory application process, thereby improving the compression efficiency.

[0064] Figure 2 This is a flow chart of the method for compressing anonymous pages proposed in the embodiment of the present disclosure. Figure 1 The embodiment shown, Figure 2 right Figure 1 Step 102 in is further described as follows: Figure 2 As shown, the following steps are included.

[0065] Step 201, determine a target compression amount of a first memory control group, and add the first memory control group and the target compression amount to a task queue.

[0066] In some embodiments, the first memory control group may be each memory control group in the process of traversing multiple memory control groups, determining a target compression amount for each memory control group, and adding each traversed memory control group and the corresponding target compression amount to a task queue.

[0067] For example, a new task queue is added. When traversing memcg in the memory recovery process, each time a memcg is traversed, a compression task and the corresponding target compression amount are submitted to the queue.

[0068] Step 202: When the task queue is not empty, determine the maximum number of threads in the thread pool based on the current first CPU load.

[0069] In some embodiments, when the task queue is not empty, that is, when there are tasks in the task queue, the upper limit of the number of threads can be determined according to the current CPU load, so that threads are allocated according to the upper limit of the number of threads in the asynchronous compression process to compress anonymous pages.

[0070] For example, a new asynchronous compression thread is added to monitor the task queue. When a task is submitted in the queue, asynchronous compression is started. The upper limit of the number of threads is updated according to the current CPU load. When the current number of threads is less than the upper limit of the number of threads, new threads are added to the thread pool until the upper limit is reached to perform memory compression.

[0071] In some embodiments, determining the maximum number of threads in the thread pool based on the current first CPU load may be determining the number of threads currently available for processing tasks based on the current CPU load, that is, the maximum number of threads is affordable by the current CPU usage.

[0072] In some embodiments, the first CPU load may be obtained by obtaining the current number of CPU cores and the current CPU usage to determine the current CPU load.

[0073] In some embodiments, the maximum number of threads in the thread pool may be the maximum number of threads used to execute tasks in the task queue. Setting the maximum number of threads may control the maximum concurrency of the thread pool and prevent the creation of too many threads from causing insufficient system resources.

[0074] In some embodiments, determining the maximum number of threads in the thread pool may be used to determine a task execution strategy in a task queue.

[0075] In some embodiments, different rules may be set for determining the maximum number of threads in the thread pool based on the first CPU load in different embodiments, which is not limited by the present disclosure.

[0076] For example, the maximum number of threads in the thread pool can be determined based on the CPU load. When the CPU load is low, for example, less than 50%, the maximum number of threads in the thread pool can be 1.5-2 times the number of CPU cores; when the CPU load is medium, for example, 50%-75%, the maximum number of threads in the thread pool can be 1-1.5 times the number of CPU cores; when the CPU load is high, for example, greater than 75%, the maximum number of threads in the thread pool can be close to or equal to the number of CPU cores.

[0077] Step 203 : for the first task in the task queue, based on the maximum number of threads, assign a first thread to compress the anonymous pages in the first memory control group.

[0078] In some embodiments, the task queue follows a first-in-first-out principle, and the task first added to the task queue will be assigned a thread first. In other words, as long as the task queue is not empty, threads will be assigned to the tasks therein in sequence.

[0079] In some embodiments, the first task in the task queue corresponds to each memory control group when traversing the memory control groups, and the first memory control group is the memory control group added to the head of the task queue according to the first-in-first-out principle.

[0080] In some embodiments, for the first task in the task queue, based on the maximum number of threads, the first thread is assigned to compress the anonymous pages in the first memory control group. When a task is added to the task queue, threads are assigned to compress the anonymous pages. The first task in the task queue is assigned threads to perform compression processing. The first task corresponds to the first memory control group traversed, and so on. In other words, each thread in the thread pool corresponds to each task in the task queue, and the number of tasks that can be processed concurrently is equal to the maximum number of threads.

[0081] For example, threads correspond to memcg one by one, and the upper limit of the current memory pool thread number is 3, so threads thread_1, thread_2, and thread_3 are assigned to perform compression tasks of memcg_1, memcg_2, and memcg_3 respectively. When a thread completes the current task, the thread is assigned to the task in the task queue.

[0082] In the above embodiment, by adding a new task queue, the target compression amount of each memory control group determined after traversing multiple memory control groups is added to the task queue, and the maximum number of threads is determined according to the CPU load, so that threads are allocated to tasks in the task queue based on the maximum number of threads, thereby achieving the purpose of asynchronous compression of anonymous pages according to the grouping of memcg, avoiding the compression process from affecting the main recovery process; and determining the maximum number of threads can also effectively control the resource consumption of the thread pool while ensuring high concurrency performance, make full use of CPU resources, and reduce resource waste.

[0083] Figure 3 A flow chart of a method for determining the maximum number of threads proposed in an embodiment of the present disclosure. Figure 1-Figure 2 The embodiment shown, Figure 3 Yes Figure 2 Further description of step 202 in FIG. Figure 3 As shown, the following steps are included:

[0084] Step 301, obtaining historical operation data and CPU quantity.

[0085] In some embodiments, the historical execution data includes the number of threads executed during the first time period.

[0086] In some embodiments, obtaining historical running data and the number of CPUs may be obtaining the number of threads and the number of CPU cores that have been historically run within a first time period.

[0087] In some embodiments, obtaining historical operation data may be obtaining the overall average number of threads of the CPU in the past first time period, for example, sampling once every 5 minutes in the first time period to determine the overall average number of threads in the first time period.

[0088] In some embodiments, obtaining the number of CPUs may be obtaining the number of CPU cores. In different embodiments, the number of CPU cores may be different. For example, the number of cores of a single-core CPU is 1, the number of cores of a dual-core CPU is 2, and the number of cores of a multi-core CPU may be 4. This disclosure does not limit this.

[0089] Step 302: Determine the average number of threads based on historical operation data and the number of CPUs.

[0090] In some embodiments, determining the average number of threads based on historical operating data and the number of CPUs may be based on the overall average number of threads of the CPU in the first time period and the number of cores of the CPU.

[0091] In some embodiments, based on the historical operation data and the number of CPUs, determining the average number of threads may be determining the value obtained by dividing the overall average number of threads in the first time period by the number of cores as the average number of threads.

[0092] For example, the basis for judging the CPU load can be the average number of threads of the CPU in the past minute, that is, sampling is performed every 5 seconds, and the value is obtained by dividing the average number of threads in the past minute by the number of CPU cores. If the value is 3, it means that there are 3 tasks being executed in the task queue of each CPU in the past minute, indicating that the CPU load is high.

[0093] Step 303: Determine the maximum number of threads based on the average number of threads.

[0094] In some embodiments, determining the maximum number of threads based on the average number of threads may be based on a preset rule, and determining the maximum number of threads based on a relationship between the average number of threads and a preset value.

[0095] In some embodiments, determining the maximum number of threads based on the average number of threads includes any of the following: when the average number of threads is less than or equal to a first threshold, determining the number of CPUs as the maximum number of threads; when the average number of threads is greater than the first threshold and less than or equal to a second threshold, determining half of the number of CPUs as the maximum number of threads; when the average number of threads is greater than the second threshold, determining the maximum number of threads as one, and the first threshold is less than the second threshold.

[0096] For example, the maximum number of threads can be determined based on the average number of threads: if the average number of threads is less than or equal to 1, the configuration upper limit is the number of CPUs; if the average number of threads is greater than 1 and less than or equal to 3, the configuration upper limit is half of the number of CPUs; if the average number of threads is greater than 3, the configuration upper limit is 1.

[0097] In the above embodiment, determining the maximum number of threads based on the historical operation data of the CPU and the number of CPUs can effectively control the resource consumption of the thread pool while ensuring high concurrency performance, fully utilize CPU resources, and reduce resource waste.

[0098] Figure 4 This is a flow chart of the memory management method proposed in the embodiment of the present disclosure. Figure 1-Figure 3 The embodiment shown, as Figure 4 As shown, the following steps are included:

[0099] Step 401: When the first thread is executed, the maximum number of threads is updated based on the current second CPU load.

[0100] In some embodiments, when the first thread is executed, that is, when a thread completes the current task, the asynchronous compression process determines the current CPU load to update the maximum number of threads in the thread pool.

[0101] In some embodiments, the current second CPU load corresponds to the time point when the thread completes executing the current task.

[0102] In some embodiments, the maximum number of threads is updated based on the current second CPU load. Figure 3 The method of determining the maximum number of threads in the thread pool is the same as shown, that is: obtaining historical operation data and the number of CPUs; determining the average number of threads based on the historical operation data and the number of CPUs; and updating the maximum number of threads based on the average number of threads.

[0103] In some embodiments, based on the average number of threads, updating the maximum number of threads can be any of the following: when the average number of threads is less than or equal to a first threshold, the number of CPUs is determined as the maximum number of threads; when the average number of threads is greater than the first threshold and less than or equal to a second threshold, half of the number of CPUs is determined as the maximum number of threads; when the average number of threads is greater than the second threshold, the maximum number of threads is determined to be one, and the first threshold is less than the second threshold.

[0104] Step 402: if the updated maximum number of threads is greater than or equal to the maximum number of threads, the second task in the task queue is assigned to the first thread, and / or the third task is assigned to the added second thread.

[0105] In some embodiments, if the updated maximum number of threads is greater than or equal to the maximum number of threads, that is, the updated number of threads remains unchanged or increases, the thread is assigned to the first task in the queue or a new thread is added and the task is assigned.

[0106] In some embodiments, the second task in the task queue may be the task currently at the head of the task queue. In other words, the second task may be the first task that needs to be compressed according to the first-in-first-out principle.

[0107] In some embodiments, when the updated maximum number of threads remains unchanged, the second task in the task queue is assigned to the first thread, that is, the thread that has completed executing the current task is assigned to the task at the head of the current task queue.

[0108] In some embodiments, when the updated maximum number of threads increases, new tasks in the task queue are assigned to the added threads. In other words, the increase in the maximum number of threads enables an increase in the number of tasks that can be processed concurrently, and the head task in the task queue that has not yet been assigned a thread can be assigned to the newly added thread.

[0109] In the above embodiment, when a thread completes its task, the maximum number of threads in the thread pool can be updated according to the CPU load, so as to reasonably set the number of threads according to the CPU load, meet the needs of the application, improve performance, and reduce resource waste.

[0110] Figure 5 This is a flow chart of the memory management method proposed in the embodiment of the present disclosure. Figure 1-Figure 4 The embodiment shown. Figure 5 As shown, the method comprises the following steps:

[0111] Step 501: If the updated maximum number of threads is less than the maximum number of threads, the first thread performs sleep.

[0112] In some embodiments, if the updated maximum number of threads is less than the maximum number of threads, the current CPU load is high, and the number of threads for concurrently processing tasks needs to be reduced to avoid affecting the CPU execution efficiency.

[0113] In some embodiments, the first thread executing sleep may be the first thread executing sleep after executing the current task and no more tasks are assigned.

[0114] Step 502, waiting for updating the maximum number of threads or destroying the first thread.

[0115] In some embodiments, the first thread performs sleep and waits for the maximum number of threads to be updated, which means that the first thread can perform sleep after completing the current task to wait for the restriction to be released. In other words, waiting for the maximum number of threads to be updated can be waiting for the maximum number of threads after the next update to exceed the current number of threads, then the first thread is released from the restriction and can continue to assign tasks to perform compression processing.

[0116] In some embodiments, the first thread executes sleep and waits for the first thread to be destroyed, which may be that the first thread executes sleep after executing the current task and is destroyed after the idle time window times out.

[0117] In some embodiments, when the first thread completes a compression task of a memory control group, if the number of threads currently created is greater than the upper limit of the maximum number of threads, the thread will no longer continue to execute subsequent compression tasks and needs to sleep and wait until the maximum number of threads is increased to exceed the existing number of threads after the next update, or it is destroyed due to the idle time window timeout. In other words, during the execution of sleep, the first thread needs to increase the updated maximum number of threads within the idle time window before the first thread can be released from the restriction. Otherwise, the first thread will be destroyed after the idle time window times out.

[0118] In the above embodiment, by updating the maximum number of threads, when the number of threads decreases, the threads that have completed the task can wait to be released from the restriction within the idle time window or be destroyed after the idle time window times out, so as to reasonably set the number of threads according to the CPU load, meet the needs of the application, and avoid waste of resources.

[0119] The following is a specific implementation of a memory management method provided by the present disclosure:

[0120] Fig. 6A A flowchart of a memory management method provided by an embodiment of the present disclosure is shown in FIG. Fig. 6A As shown, the method comprises the following steps:

[0121] 1. When a new application requests memory, determine whether it can directly request free memory;

[0122] If free memory can be directly applied for, the application is completed and memory is allocated to the application;

[0123] like Figure 6B The schematic diagram of judging memory recycling method is shown in the figure. If free memory cannot be directly applied for, judge whether the current zone meets the low watermark:

[0124] If it is satisfied, it determines whether it meets the min waterline. If it is satisfied, it determines whether free memory can be directly applied for. If it does not meet the min waterline, it executes the direct recycling process. If the current zone does not meet the low waterline, it wakes up kswapd and executes the asynchronous recycling process.

[0125] 2. Calculate the target recovery amount

[0126] 3. Determine whether the target recycling amount has been collected: If the target recycling amount has been collected, execute step 1 to determine whether free memory can be directly applied for.

[0127] 4. If the target recovery amount has not been collected yet, traverse memcg, get iru, and calculate the recovery ratio of file pages and anonymous pages.

[0128] 5. Submit anonymous page compression to the task queue and execute the asynchronous compression process.

[0129] The schematic diagram of the asynchronous compression process is as follows Figure 6C As shown, the following steps are included:

[0130] 5.1. Initialize the asynchronous compression thread at startup;

[0131] 5.2. Initialize the task queue;

[0132] 5.3. Thread sleep, if there is no task within the time window, the redundant threads will be destroyed;

[0133] 5.4. Determine whether there are tasks to be processed in the task queue: If there are no tasks, the thread sleeps and executes 5.3;

[0134] 5.5. When there are tasks, set the upper limit of the number of threads according to the current CPU load;

[0135] 5.6. Determine whether the current number of threads is less than the upper limit of the number of threads;

[0136] 5.7. If the current number of threads is less than the upper limit of the number of threads, add new threads to the upper limit of the number of threads;

[0137] 5.8. If the current number of threads is greater than or equal to the upper limit of the number of threads, perform memory compression: assign threads to each submitted task for execution, and limit the number of threads according to the current system load. The schematic diagram of the allocation strategy is as follows: Fig.6DAs shown, the designated threads correspond to memcg one by one. For example, if the current upper limit of the number of threads is 3, threads thread_1, thread_2, and thread_3 are assigned to perform the compression tasks of memcg_1, memcg_2, and memcg_3 respectively. When a thread completes the current task, the asynchronous thread determines the current system CPU load and updates the upper limit of the number of threads. If the updated number remains unchanged or increases, the thread is assigned to the first task in the queue or a new thread is added and the task is assigned. If the updated number decreases, the thread sleep waiting limit is lifted or destroyed.

[0138] 6. File page write back / demap.

[0139] The beneficial theoretical effects of the memory management method proposed in this disclosure are:

[0140] 1. The latency of kswapd and dr is reduced, and the application memory application process is not blocked;

[0141] 2. The compression speed is greatly improved compared with the serial method.

[0142] Figure 7 FIG. 7 is a schematic diagram of the structure of the memory management device 700 according to an embodiment of the present disclosure. Figure 7 As shown, the device comprises:

[0143] The first processing module 701 is used to start the asynchronous compression process when the current free memory is lower than the waterline value.

[0144] The second processing module 702 is used to traverse multiple memory control groups and allocate threads to each memory control group through an asynchronous compression process to perform compression processing on anonymous pages in each memory control group.

[0145] In some embodiments, the first processing module is also used to start an asynchronous recovery process if the current first free memory is greater than the second watermark value and less than the first watermark value; if the current second free memory is less than or equal to the second watermark value, start a direct recovery process, and the asynchronous recovery process and the direct recovery process are used to recycle file pages in each memory control group, and the second watermark value is less than the first watermark value.

[0146] In some embodiments, the second processing module is also used to determine the target compression amount of the first memory control group, and add the first memory control group and the target compression amount to the task queue; when the task queue is not empty, determine the maximum number of threads in the thread pool based on the current first CPU load; for the first task in the task queue, based on the maximum number of threads, assign the first thread to compress the anonymous pages in the first memory control group.

[0147] In some embodiments, the second processing module is also used to obtain historical operation data and CPU number, the historical operation data including the number of threads running in the first time period; based on the historical operation data and the CPU number, determine the average number of threads; based on the average number of threads, determine the maximum number of threads.

[0148] In some embodiments, the second processing module is also used to update the maximum number of threads based on the current second CPU load when the first thread is executed; if the updated maximum number of threads is greater than or equal to the maximum number of threads, the second task in the task queue is assigned to the first thread, and / or the third task is assigned to the added second thread.

[0149] In some embodiments, the second processing module is also used to determine the number of CPUs as the maximum number of threads when the average number of threads is less than a first threshold; when the average number of threads is greater than the first threshold and less than a second threshold, determine half of the number of CPUs as the maximum number of threads; when the average number of threads is greater than the second threshold, determine the maximum number of threads as one, and the first threshold is less than the second threshold.

[0150] In some embodiments, the second processing module is further configured to, if the updated maximum number of threads is less than the maximum number of threads, cause the first thread to sleep; wait for the maximum number of threads to be updated or for the first thread to be destroyed.

[0151] In summary, the memory management device proposed in the present disclosure starts an asynchronous compression process through the first processing module when the current free memory is lower than the waterline value; the second processing module traverses multiple memory control groups and allocates threads to each memory control group through the asynchronous compression process to compress the anonymous pages in each memory control group. The device provided by the present disclosure improves the compression efficiency by asynchronousizing the anonymous page compression part in the memory recovery process, creating a thread pool and performing parallel compression according to the memcg memory control group. At the same time, the asynchronous memory compression process does not affect the main recovery process. It solves the problem that the entire recovery process is blocked due to the long compression time of anonymous pages, thereby affecting the recovery efficiency.

[0152] Figure 8 is a structural diagram of an electronic device 800 for implementing the above-mentioned memory management method according to an exemplary embodiment.

[0153] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , an input / output (I / O) interface 808 , a sensor component 810 , and a communication component 812 .

[0154] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, battery management, and logging operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a balancing module to facilitate interaction between the power component 806 and the processing component 802.

[0155] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0156] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0157] I / O interface 808 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0158] The sensor assembly 810 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 810 can detect the open / closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 810 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 810 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 810 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 810 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0159] The communication component 812 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 812 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 812 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0160] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0161] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by the processor 820 of the electronic device 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0162] An embodiment of the present disclosure further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the memory management method described in any one of the above embodiments of the present disclosure.

[0163] An embodiment of the present disclosure also proposes a chip, comprising at least one processor and a communication interface; the communication interface is used to receive signals input into the chip or signals output from the chip, the processor communicates with the communication interface and implements the memory management method described in the above embodiment of the present disclosure through logic circuits or execution code instructions.

[0164] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0165] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0166] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0167] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0168] It should be understood that the various parts of the embodiments of the present disclosure can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0169] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0170] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0171] Although the embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present disclosure. A person skilled in the art may change, modify, substitute and vary the above embodiments within the scope of the present disclosure.

Claims

1. A memory management method, characterized in that: The method comprises: When the current free memory is lower than the waterline value, start the asynchronous compression process; A plurality of memory control groups are traversed, and a thread is allocated to each memory control group through the asynchronous compression process to perform compression processing on the anonymous pages in each memory control group.

2. The method according to claim 1, characterized in that When the current free memory is lower than the waterline value, the method further includes at least one of the following: If the current first free memory is greater than the second watermark value and less than the first watermark value, start the asynchronous recycling process; If the current second free memory is less than or equal to the second watermark value, start the direct recovery process, the asynchronous recovery process and the direct recovery process are used to recycle the file pages in each memory control group, and the second watermark value is less than the first watermark value.

3. The method according to claim 2, characterized in that The traversing of the plurality of memory control groups and assigning a thread to each memory control group through the asynchronous compression process to compress the anonymous pages in each memory control group includes: Determining a target compression amount for a first memory control group, and adding the first memory control group and the target compression amount to a task queue; When the task queue is not empty, determining a maximum number of threads in the thread pool based on the current first CPU load; For the first task in the task queue, based on the maximum number of threads, a first thread is allocated to compress the anonymous pages in the first memory control group.

4. The method according to claim 3, characterized in that Determining the maximum number of threads in the thread pool based on the current first CPU load includes: Obtaining historical running data and the number of CPUs, wherein the historical running data includes the number of threads running in a first time period; Determine an average number of threads based on the historical operation data and the number of CPUs; Based on the average number of threads, the maximum number of threads is determined.

5. The method according to claim 4, characterized in that The method further comprises: When the first thread is executed, based on the current second CPU load, updating the maximum number of threads; If the updated maximum number of threads is greater than or equal to the maximum number of threads, the second task in the task queue is allocated to the first thread, and / or the third task is allocated to the added second thread.

6. The method according to claim 4 or 5, characterized in that: The method further comprises any of the following: When the average number of threads is less than or equal to a first threshold, determining the number of CPUs as the maximum number of threads; When the average number of threads is greater than the first threshold and less than or equal to the second threshold, half of the number of CPUs is determined as the maximum number of threads; When the average number of threads is greater than the second threshold, the maximum number of threads is determined to be one, and the first threshold is less than the second threshold.

7. The method according to claim 5, characterized in that The method further comprises: If the updated maximum number of threads is less than the maximum number of threads, the first thread performs sleep; Waiting for updating the maximum number of threads or destroying the first thread.

8. A memory management device, characterized in that: include: The first processing module is used to start the asynchronous compression process when the current free memory is lower than the waterline value; The second processing module is used to traverse multiple memory control groups and allocate threads to each memory control group through the asynchronous compression process to perform compression processing on the anonymous pages in each memory control group.

9. An electronic device, characterized in that: include: a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the method according to any one of claims 1 to 7.

10. A computer storage medium, wherein: The computer storage medium stores computer executable instructions; after the computer executable instructions are executed by the processor, the method described in any one of claims 1 to 7 can be implemented.

11. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

12. A chip, characterized in that: It includes at least one processor and a communication interface; the communication interface is used to receive a signal input to the chip or a signal output from the chip, and the processor communicates with the communication interface and implements the method as described in any one of claims 1-7 through a logic circuit or executing code instructions.

Images